Good Teaching, Bad Results

The past decade has seen a radical shift in theories of learning, brought about in large part by progress in the cognitive sciences. Through perhaps the mid-1970's, learning theories were for the most part domain-independent. Such theories attempted to characterize general principles of learning, the specifics of which were hypothetically applicable in different domains such as reading, social studies, and mathematics. The details of the subject matter were not important in such theories, for the most part playing the role of "context variables" that (as the theory had it) could be taken into account in experimental design. The corresponding paradigms for investigating research on teaching are described by Corno (this volume); see also Doyle (1978), Dunkin and Biddle (1974), and Shulman (1985). The first major paradigm described by Corno, process-product research, largely used correlational methods to explore relationships between teacher classroom behavior and student learning. The classroom behaviors explored were for the most part straightforward and easily quantifiable: e.g. time spent in questioning, "active learning time," amount of praise, amount of feedback. Other classroom variables included type of ability grouping, whether students worked in small or large groups, and so on. Learning was operationally defined as performance on achievement tests -- tests which, as we shall see below, may fail in significant ways to measure subject matter understanding.

Mediating process research (see, e.g., Corno, this volume) provides a means of overcoming some of the significant limitations of the process-product paradigm. Such work signals the beginning of a rapprochement with cognitive science research on learning, specifically with its focus on the child as active interpreter of its experience. Doyle's study in this volume provides some compelling examples of the importance of this perspective. Doyle suggests that the presentation of subject matter as familiar work  routinized exercises that can be worked out of context, and without significant understanding of the subject matter -- can trivialize that subject matter and deprive students of the opportunity to understand and use what they have studied. That suggestion is explored at length in this paper.

Recent cognitive research on learning diverges from the domain-independent work described above in that it lays a much greater emphasis on the particulars of the subject matter being studied. In elementary arithmetic, for example, Brown and Burton (1978) developed a diagnostic test that could predict, about 50% of the time, the incorrect answers that a particular student would obtain to a subtraction problem -- before the student worked the problem! The literature indicates that misconceptions in arithmetic, in algebra, in physics, and other domains, are quite common and consistent (see, e.g., Helms & Novak, 1985.). From this and related work follow two main consequences.

The first consequence is that one of the treasured pedagogical principles on which much current instruction is based is, if not plain wrong, certainly inadequate. The predominant model of current instruction is based on what Romberg and Carpenter (1985) call the absorbtion theory of learning. "The traditional classroom focuses on competition, management, and group aptitudes; the mathematics taught is assumed to be a fixed body of knowledge, and it is taught under the assumption that learners absorb what has been covered" (p. 26). According to this view, the good teacher is the one who has ten different ways to say the same thing; the student is sure to "get it" sooner or later. However, the misconceptions literature indicates that the students may well have "gotten" something else -- and that what the student has gotten may be resistant to change. Dealing with this reality calls for a significantly different perspective on the part of the teacher. It also calls for different perspectives regarding the appropriate domain of study of research on teaching, and different measures of competence. The second consequence is that one must look at the subject matter in detail. Arithmetic mistakes differ from misconceptions in algebra and physics, and from misapprehensions about reading; we will understand each of these only by studying it on its own terms. Thus studies of learning and teaching in particular subject areas must be grounded in analyses of what it means to understand the subject matter being taught. It is to that kind of analysis, in mathematics, that we now turn. Some relevant research on mathematical cognition and teaching may be found in Romberg and Carpenter (1985), Leinhardt & Smith (1984), Resnick (1983), Schoenfeld (1985), and Silver (1985).

The issue of classroom practice and its relation to students' understanding of mathematical structure was one of the main themes of Wertheimer's (1959) Productive Thinking, which provides our first two examples. In the first, Wertheimer asked elementary school students to solve problems.  Many of the students, who were fluent in all four of the basic arithmetic operations, solved such problems by laboriously adding the terms in the numerator and then performing the indicated division. By virtue of obtaining the correct answer, the students indicated that they had mastered the procedures of the discipline. However, they had clearly not mastered the underlying substance; if you see repeated addition as equivalent to multiplication and you see division as the inverse of multiplication (i.e., the multiplication and division by the same number cancel each other out), there is no need to calculate at all. This example illustrates that being able to perform the appropriate algorithmic procedures, while important, does not necessarily indicate any depth of understanding. (We note here that virtually all standardized testing for arithmetic competency -- and, de facto, much standard instruction in arithmetic -- focuses primarily if not exclusively on procedural mastery.)

Wertheimer's more famous example comes from his observations of classroom sessions devoted the "the parallelogram problem," the problem of determining the area of a parallelogram of base B and altitude H. The students had been taught the standard procedure, where cutting off and moving a specific triangle converts the parallelogram to a rectangle whose area is easy to calculate. They did quite well at the lesson, and they were able to reproduce the argument in mathematically correct form. But when Wertheimer asked the students to find the area of a parallelogram in non-standard position, or to find the area of a parallelogram-like figure to which the same argument applied, the students were stymied. Wertheimer argues that although they had memorized the proof, they had failed to understand the reason that it worked; although

they had memorized the formula, they used it without deep understanding. With that understanding, he argues, the students would have been able to answer his questions without difficulty; without it they could solve certain well specified exercises but in reality had acquired only the superficial appearance of competence. (We note again that typical achievement tests, which examine students' ability to reproduce the standard arguments, are unlikely to examine the kinds of understandings Wertheimer considers fundamental.)

There are numerous contemporary parallels to these examples. For example, word problems of the following type are a major focus of the elementary mathematics curriculum: "John has 8 apples. He gives 5 apples to Mary. How many apples does John have left?" Perhaps the most commonly used instructional procedure to help students solve such problems is the "key word procedure," which is used as follows. The student is told that certain words in problem statements provide the "key" to selecting which arithmetic operation to employ. For example, the key word in the problem just quoted is left, which indicates subtraction. One can "solve" the problem by identifying the two numbers in the problem statement, and then -- since the key word is "left" -- subtracting one from the other. Note that one can do so without even reading the whole problem, and without understanding the situation it describes. Research indicates that many students work the problems in precisely that fashion. In interviews some students revealed that they circled the numbers in the problem statement and then read the problem statement from the last word backwards, because the key word usually appears near the end of the problem! Thus the key word procedure, initially introduced to help students make sense of word problems, had (at least in these cases) precisely the opposite effects. It allowed students to obtain the right answers without understanding -- and gave them the option of not seeking understanding at all. Worse, it may have suggested to them that understanding is not necessary when solving mathematics problems; one simply follows the procedure, whether it makes sense or not.

The most extensive documentation of students' performance on word problems, without understanding, comes from the third National Assessment of Educational Progress (Carpenter, Lindquist, Matthews, and Silver, 1983). On the NAEP mathematics exam, which used a stratified national sample of 45,000 students, 13-year-olds were given the following problem: "An army bus holds 36 soldiers. If 1128 soldier are being bussed to their training site, how many buses are needed?" Seventy percent of the students who worked the problem performed the long division algorithm correctly. However, 29% of the students wrote that the number of buses needed is "31 remainder 12" and another 18% wrote that the number of buses needed is 31. Only 23% gave the correct answer. Thus fewer than one-third of the students who selected and carried out the appropriate algorithm produced the right answer -- a step that required a trivial analysis of the meaning of the problem statement. There are a number of plausible explanations for this behavior, one of which will be suggested in the case study below (See also Silver, in press, for a discussion of related problems.). But data of this type document an almost universal phenomenon: Students who are capable of performing symbolic operations in a classroom context, demonstrating "mastery" of certain subject matter, often fail to map the results of the symbolic operations they have performed to the systems that have been described symbolically. That they fail to connect their formal symbol manipulation procedures with the "real world" objects represented by the symbols constitutes a dramatic failure of instruction.

A set of similar phenomena motivated the present study. I have conducted a series of studies exploring students' understandings of geometry. Those studies have focused, in particular, on the relationship between geometric proofs and geometric constructions. To sum things up briefly, I had found that high school and college students who had taken a full year of high school geometry -- which focuses on proving theorems about geometric objects -- uniformly approached geometric construction problems as empiricists. They engaged in empirical guess-and-test loops, completely ignoring their proof-related knowledge. In one series of interviews, for example, college students were asked to work two related problems. The first was a proof problem. Solving this problem directly provided the answer to the second, a construction problem (The second problem asked how to construct a circle whose properties had been completely determined in the first.). Yet, after solving the first problem, nearly a third of the students began the second problem by making conjectures that flatly violated the results they had just proved!

Such behavior indicated that these students saw little or no connection between their "proof knowledge," abstract mathematical knowledge about geometric figures obtained by formal deductive means, and their "construction knowledge," procedures and information they had mastered in the very same class for working straightedge and compass construction problems. I make this statement more provocatively as Belief 1, below; some other typical beliefs are also given. I conjecture students may develop these beliefs as a result of their experiences with mathematics. (Extended discussions of the students' beliefs may be found in chapters 5 and 10 of my (1985) Mathematical Problem Solving. A discussion of the "ideal" relationship between geometric empiricism and deduction may be found in Schoenfeld, in press.)

Belief 1: The processes of formal mathematics (e.g. "proof") have little or nothing to do with discovery or invention. Corollary: Students fail to use information from formal mathematics when they are in "problem solving mode."

Belief 2: Students who understand the subject matter can solve assigned mathematics problems in five minutes or less. Corollary: Students stop working on a problem after just a few minutes since, if they haven't solved it, they didn't understand the material (and therefore will not solve it.)

Belief 3: Only geniuses are capable of discovering, creating, or really understanding mathematics. Corollary: Mathematics is studied passively, with students accepting what is passed down "from above" without the expectation that they can make sense of it for themselves. In listing these beliefs we note the parallel to research Doyle describes in this volume. Doyle described a student who took a teacher's instructions for an assignment as a recipe for completing the task, rather than a way of learning the material. In terms more provocative than Doyle might like, one can characterize that student's perspective as follows:

Belief 4: One succeeds in school by performing the tasks, to the letter, as described by the teacher. Corollary: learning is an incidental by-product to "getting the work done."

[S]tudents may not understand some of the problems they do solve. Most of the routine problems can be mechanically solved by applying a routine computational algorithm. In such problems the students may have no need to understand the problem situation, why the particular computation is appropriate, or whether the answer is reasonable... The errors made on several of the problems indicate that students generally try to use all of the numbers given in a problem statement in their calculation, without regard for the relationship of either the given numbers or the resulting answers to the problem situation. (Carpenter et al., 1983, p. 656)

In sum, research on the teaching and learning needs to be expanded both in scope and in breadth. "Learning outcomes" must be broadly defined if we are to provide adequate characterizations of behaviors such as those described in the previous paragraph. But explorations of learning also need to become more focused and detailed as we begin to elaborate on what it means to think mathematically. It is also essential -- both for research purposes and because measurement is the "bottom line" for much real world instruction -- for our efforts to include the development of measures that will adequately characterize this expanded notion of mathematics learning. And if we really intend to affect practice, we will need to become deeply involved in the development and testing of instructional materials. This list of tasks may seem daunting, but it is not beyond our reach. There is, as noted above, an increasing rapprochement between researchers on teaching and cognitive scientists. Similarly, there are closer ties between psychologists of learning and subject matter experts as the result of perceived need for collaborative efforts. As our sense of the task grows, so does our capacity to deal with it.