Discussions about feedback frequently take place within a framing of assessment and student achievement (Black & Wiliam, 1998; Boud, 2000). In this context, the primary role of feedback is to help the student address any perceived deficits as identified through the completion of an assessment item. Ironically, assessment scores and student achievement data have also become tools for driving political priorities and agendas, and are also used as indicators in quality assurance requirements.
Assessment in essence is a two-edged sword used to foster learning as well as a tool for measuring quality assurance and establishing competitive rankings (Wiliam, Lee, Harrison, & Black, 2004). While acknowledging the importance of assessment for quality assurance, we focus specifically on the value of feedback often associated with formative assessment or simply as a component of student completion of set learning tasks. Thus, this article explores how student trace data can be exploited to facilitate the transformation of the essence of assessment practices by focusing on feedback mechanisms. With such a purpose, I highlight and discuss current approaches to the creation and delivery of data-enhanced feedback as exemplified through the vast body of research in learning analytics and educational data mining (LA/EDM).
Although there is no unified definition of feedback in educational contexts, several comprehensive analyses of its effects on learning have been undertaken (e.g., Evans, 2013; Hattie & Timperley, 2007; Kluger & DeNisi, 1996). In sum, strong empirical evidence indicates that feedback is one of the most powerful factors influencing student learning (Hattie, 2008). The majority of studies have concluded that the provision of feedback has positive impact on academic performance. However, the overall effect size varies and, in certain cases, a negative impact has been noted.
For instance, a meta-analysis by Kluger and DeNisi (1996) demonstrated that poorly applied feedback, characterized by an inadequate level of detail or the lack of relevance of the provided information, could have a negative effect on student performance. In this case, the authors distinguished between three levels of the locus of learner’s attention in feedback: the task, the motivation, and the meta-task level. All three are equally important and can vary gradually in focus. Additionally, Shute (2008) classified feedback in relation to its complexity, and analyzed factors affecting the provision of feedback such as its potential for negative impact, the connection with goal orientation, motivation, the presence in scaffolding mechanisms, timing, or different learner achievement levels. Shute noted that to maximize impact, any feedback provided in response to a learner’s action should be non-evaluative, supportive, timely, and specific.
Early models relating feedback to learning largely aimed to identify the types of information provided
to the student. Essentially, these studies sought to characterize the effect that different types of information can play on student learning (Kulhavy & Stock, 1989). Initial conceptualizations of feedback were driven by the differences in learning science theorisations of how the gap between the actual and desired state of the learner can be bridged (cf. historical review Kluger & DeNisi, 1996; Mory, 2004). According to Mory (2004), contemporary models build upon pre-existing paradigms by viewing feedback in the context of self-regulated learning (SRL), i.e., a style of engaging with tasks in which students exercise a suite of powerful skills (Butler & Winne, 1995). These skills, setting goals, thinking about strategies, selecting the right strategies, and monitoring the effects of these strategies on the progress towards the goals are all associated with student achievement (Butler & Winne, 1995; Pintrich, 1999; Zimmerman, 1990). As part of their theoretical synthesis between feedback and self-regulated learning, Butler and Winne (1995, p. 248) embedded two feedback loops into their model. The first loop is contained within the so-called cognitive system and refers to the capacity of individuals to monitor their internal knowledge and beliefs, goals, tactics, and strategies and change them as required by the learning scenario. The second loop occurs when the product resulting from a student engaging with a task is measured, prompting the creation of external feedback relayed back to the student; for example, an assessment score, or an instructor commenting upon the completion of a task.
Hattie and Timperley (2007) have provided one of the most influential studies on feedback and its impact on achievement. The authors’ conceptual analysis was underpinned by a definition of feedback as the information provided by an agent regarding the performance or understanding of a student. The authors proposed a model of feedback articulated around the concept that any feedback should aim to reduce the discrepancy between a student’s current understanding and their desired learning goal. As such, feedback can be framed around three questions: where am I going, how am I going, and where to next? Hattie and Timperley (2007) proposed that each of these questions should be applied to four different levels: learning task, learning process, self-regulation, and self. The learning task level refers to the elements of a simple task; for example, notifying the student if an answer is correct or incorrect. The learning process refers to general learning objectives, including various tasks at different times. The self-regulation level refers to the capacity of reflecting on the learning goals, choosing the right strategy, and monitoring the progress towards those goals. Finally, the self level refers to abstract personality traits that may not be related to the learning experience.
The process and regulation levels are argued to be the most effective in terms of promoting deep learning and mastery of tasks. Feedback at the task-level is effective only as a supplement to the previous two levels; feedback at the self-level has been shown to be the least effective. These three questions and four levels of feedback provide the right setting to connect feedback with other aspects such as timing, positive vs. negative messages (also referred to as polarity), and the consequences of including feedback as part of an assessment instrument. These aspects have been shown to have a interdependent effect that can be positive or negative (Nicol & Macfarlane-Dick, 2006).
In reviewing established feedback models, Boud and Molloy (2013) argued that they are at times based on unrealistic assumptions about the students and the educational setting. Commonly, due to resource constraints, the proposed feedback models or at least the mechanism for generating non-evaluative, supportive, timely, and specific feedback for each student is impractical or at least not sustainable in contemporary educational scenarios. At this juncture, LA/EDM work can play a significant role in moving feedback from an irregular and unidirectional state to an active dialogue between agents.
The first initiatives using vast amounts of data to improve aspects of learning can be traced to areas such as adaptive hypermedia (Brusilovsky, 1996; Kobsa, 2007), intelligent tutoring systems (ITSs) (Corbett, Koedinger, & Anderson, 1997; Graesser, Conley, & Olney, 2012), and academic analytics (Baepler & Murdoch, 2010; Campbell, DeBlois, & Oblinger, 2007; Goldstein & Katz, 2005). Much of this research has taken place within LA/EDM research communities that share a common interest in data-intensive approaches to the research of educational setting, with the purpose of advancing educational practices (Baker & Inventado, 2014). While these communities have many similarities, there are some acknowledged differences between LA and EDM (Baker & Siemens, 2014). For example, EDM has a more reductionist focus on automated methods for discovery, as opposed to LA’s human-led explorations situated within holistic systems. Baker and Inventado (2014) noted that the main differences between LA and EDM are not so much in the preferred methodologies, but in the focus, research questions, and eventual use of models.
When considering LA/EDM through the lens of feedback, the research approaches differ in relation to the direction and recipient of feedback. For instance, LA initiatives generally provide feedback aimed towards developing the student in the learning process (e.g., self-regulation, goal setting, motivation, strategies, and tactics). In contrast, EDM initiatives tend to focus on the provision of feedback to address changes in the learning environment (e.g., providing hints that modify a task, recommending heuristics that populate the environment with the relevant resources, et cetera).
It is important to note that these generalizations are not a hard categorization between the communities, more so an observed trend in LA/EDM works that reflects their disciplinary backgrounds and interests. The following section further unpacks the work in both the EDM and LA communities related to the provision of feedback to aid student learning.
No comments:
Post a Comment