Learning Analytics

Defining Learning Analytics 

“Learning analytics refers to the interpretation of a wide range of data produced by and gathered on behalf of students in order to assess academic progress, predict future performance, and spot potential issues. Data are collected from explicit student actions, such as completing assignments and taking exams, and from tacit actions, including online social interactions, extracurricular activities, posts on discussion forums, and other activities that are not directly assessed as part of the student’s educational progress. Analysis models that process and display the data assist faculty members and school personnel in interpretation. The goal of learning analytics is to enable teachers and schools to tailor educational opportunities to each student’s level of need and ability.” “Learning analytics need not simply focus on student performance. It might be used as well to assess curricula, programs, and institutions. It could contribute to existing assessment efforts on a campus, helping provide a deeper analysis, or it might be used to transform pedagogy in a more radical manner. It might also be used by students themselves, creating opportunities for holistic synthesis across both formal and informal learning activities.” 

Learning analytics is becoming defined as an area of research and application and is related to academic analytics, action analytics, and predictive analytics. 1 Learning analytics emphasizes measurement and data collection as activities that institutions need to undertake and understand, and focuses on the analysis and reporting of the data. Unlike educational data mining, learning analytics does not generally address the development of new computational methods for data analysis but instead addresses the application of known methods and models to answer important questions that affect student learning and organizational learning systems. 

The goal of learning analytics as enabling teachers and schools to tailor educational opportunities to each student’s level of need and ability. Unlike educational data mining, which emphasizes system generated and automated responses to students, learning analytics enables human tailoring of responses, such as through adapting instructional content, intervening with atrisk students, and providing feedback. Learning analytics draws on a broader array of academic disciplines than educational data mining, incorporating concepts and techniques from information science and sociology, in addition to computer science, statistics, psychology, and the learning sciences. Unlike educational data mining, learning analytics generally does not emphasize reducing learning into components but instead seeks to understand entire systems and to support human decision making. 

Technical methods used in learning analytics are varied and draw from those used in educational data mining. Additionally, learning analytics may employ: 

• Social network analysis (e.g., analysis of student-tostudent and student-to-teacher relationships and interactions to identify disconnected students, influencers, etc.) and 

• Social or “attention” metadata to determine what a user is engaged with. As with educational data mining, providing a visual representation of analytics is critical to generate actionable analyses; information is often represented as “dashboards” that show data in an easily digestible form. 

A key application of learning analytics is monitoring and predicting students’ learning performance and spotting potential issues early so that interventions can be provided to identify students at risk of failing a course or program of study. Several learning analytics models have been developed to identify student risk level in real time to increase the students’ likelihood of success. Educational institutions have shown increased interest in learning analytics as they face calls for more transparency and greater scrutiny of their student recruitment and retention practices. Data mining of student behavior in online courses has revealed differences between successful and unsuccessful students (as measured by final course grades) in terms of such variables as level of participation in discussion boards, number of emails sent, and number of quizzes completed. Analytics based on these student behavior variables can be used in feedback loops to provide more fluid and flexible curricula and to support immediate course alterations (e.g., sequencing of examples, exercises, and self-assessments) based on analyses of real-time learning data. 

In summary, learning analytics systems apply models to answer such questions as: 

• When are students ready to move on to the next topic? 

• When are students falling behind in a course? 

• When is a student at risk for not completing a course? 

• What grade is a student likely to get without intervention? 

• What is the best next course for a given student? 

• Should a student be referred to a counselor for help? 

Learning Analytics Applications 

Educational data mining and learning analytics research are beginning to answer increasingly complex questions about what a student knows and whether a student is engaged. For example, questions may concern what a short-term boost in performance in reading a word says about overall learning of that word, and whether gaze-tracking machinery can learn to detect student engagement. Researchers have experimented with new techniques for model building and also with new kinds of learning system data that have shown promise for predicting student outcomes. 

The application areas were discerned from the review of the published and gray literature and were used to frame the interviews with industry experts. These areas represent the broad categories in which data mining and analytics can be applied to online activity, especially as it relates to learning online. This is in contrast to the more general areas for big data use, such as health care, manufacturing, and retail. These application areas are 

(1) modeling of user knowledge, user behavior, and user experience; 

(2) user profiling; 

(3) modeling of key concepts in a domain and modeling a domain’s knowledge components, 

(4) and trend analysis. 

Another application area concerns how analytics are used to adapt to or personalize the user’s experience. Each of these application areas uses different sources of data, and describes questions that these categories answer and lists data sources that have been used thus far in these applications.

New technology start-ups founded on big data (e.g., Knewton, Desire2Learn) are optimistic about applying data mining and analytics—user and domain modeling and trend analysis—to adapt their online learning systems to offer users a personalized experience. Companies that “own” personal data (e.g., Yahoo!, Google, LinkedIn, Facebook) have supported open-source developments of big data software (e.g., Apache Foundation’s Hadoop) and encourage collective learning through public gatherings of developers to train them on the use of these tools (called hackdays or hackathons). The big data community is, in general, more tolerant of public trialand-error efforts as they push data mining and analytics technology to maturity.

The challenges in implementing data mining and learning analytics within K–20 settings. Experts pose a range of implementation considerations and potential barriers to adopting educational data mining and learning analytics, including technical challenges, institutional capacity, legal, and ethical issues. Successful application of educational data mining and learning analytics will not come without effort, cost, and a change in educational culture to more frequent use of data to make decisions. What is the gap between the big data applications in the commerce, social, and service sectors and K–20 education? Given that learning analytics practices have been applied primarily in higher education thus far, the time to full adoption may be longer in different educational settings, such as K–12 institutions.

Education institutions pioneering the use of data mining and learning analytics are starting to see a payoff in improved learning and student retention. As described student data can help educators both track academic progress and understand which instructional practices are effective. How students can examine their own assessment data to identify their strengths and weaknesses and set learning goals for themselves. Recommendations from this guide are that K–12 schools should have a clear strategy for developing a data-driven culture and a concentrated focus on building the infrastructure required to aggregate and visualize data trends in timely and meaningful ways, a strategy that builds in privacy and ethical considerations at the beginning. The vision that data can be used by educators to drive instructional improvement and by students to help monitor their own learning is not new. However, the feasibility of implementing a data-driven approach to learning is greater with the more detailed learning micro data generated when students learn online, with newly available tools for data mining and analytics, with more awareness of how these data and tools can be used for product improvement and in commercial applications, and with growing evidence of their practical application and utility in K–12 and higher education. There is also substantial evidence of effectiveness in other areas, such as energy and health care.