Designing of Assessment items - Context and Format

Contexts. 

One should not fall into the tempting trap of designing items with the mathematics content in mind and then later adding a context. Nor should one take a context problem and simply change the context—many examples are available to show the disasters that can happen when these design strategies are followed.

The distance of the context to the students’ real world is one aspect that the teacher needs to get under control in the sense that each class environment can generate its own rules. (The assessment contract plays a crucial role here.) If the teacher takes the news from the different media as a regular discussion point in the lessons, one might expect a greater spread of distances in contexts than is the case with a teacher who facilitates contexts especially close to home and school. There is a clear trend for younger students to feel more confident with contexts closer to their life; to the surprise of some, however, context that relates to the environment, nature, and daily life in a wider sense can also motivate students very well, assuming challenging problems. One should also be aware that in the more informal assessment formats, the freedom in context use is greater than when the tests are of a more summative character. This is because a teacher in a discussion will immediately note whether a certain context seems sensible to certain students (certain illnesses, for instance), and the teacher can correct for that on the spot.

The relevance of the context is another important point of consideration. If we take problem solving, and thus mathematization, as an important aspect of mathematics, then it is absolutely necessary to include first- and preferably second-order contexts on a regular basis. Quite often this means the use of more complex formats, although extended-response written questions offer good possibilities.

Finally, the point of “realness” of the context needs attention. Here again the teacher, in relation with the students, sets the boundaries. Although it seems sensible to stay as close to reality as possible without losing mathematical relevance, there are also good examples of not so-real or not-so-authentic problems that have been excellent items for assessment, along with a few almost ridiculous “fantasy” problems that functioned within the classroom environment defined by teacher and students.

Formats.

It is too simple to state that the choice of formats should be balanced. When we have a learning trajectory, one cannot just identify crucial assessment opportunities and choose a format. But in general, one can say certain things about the choice. It will be evident that discourse and observations are the continuous mode of operation together with homework. What is not trivial is that this has to be carried out with some structure and focus on key concepts. Also, some thought needs to be given to keeping track of the “scores” of the students on these formats. Again, technology can support us a bit with these issues: Handheld PDIs with dedicated software can be a considerable help in tracking the observations.

Regularly, some kind of restricted-time written test will be the backbone of our system. Assuming we have the right quality of items, there’s nothing wrong with that. These can vary in time from a short, 10-minute quiz to a two hour–long problem-solving task. We also need to stress more “constructive” formats of assessment with some mode of a two-stage task that can fit in well, although certainly not too often—maybe at most three times per year. Part of the minimal requirements for a representative assessment system include that self assessment be systemic and that homework should function, at least in part, as assessment.

It seems advisable to construct a path of incremental change in relation to more challenging assessment formats. The design, or even a proper judgment of open–open ended questions is already a very complex task. And although it seems sometimes easier to design a project task (like the question: “Is the pattern of green and red lights at this intersection the optimal in relation to the traffic flow?”), problems abound about such concerns as the execution, logistics, level of individuality, data sampling, and reporting in and out of school, not to mention how to cope with the different reports when the scoring, grading, and feedback are to be discussed. One should be careful not to fall into the hole of entering a very promising but extremely complex area of the assessment landscape without the prior experience of closely related formats.