Jacqueline Wong
The use of learning analytics to identify and support students who are at risk of underperforming or dropping out is becoming increasingly popular among higher education institutions. Even though past research appears to support the predictive power of learning analytics models, little is known about the effectiveness of learning analytics interventions. Larrabee Sønderlund, Hughes, and Smith (2018) found only 11 studies out of 689 identified papers to have evaluated effectiveness of interventions involving learning analytics. The 11 studies examined the use of learning analytics to enhance student grades and retention in a similar manner: learning analytics was used to identify at-risk students. After identifying the at-risk students and analysing the issues, the students or the teachers were then informed and expected to become more aware of the learning issue, remediate the issue, and ultimately, achieve higher academic success and retention. However, the design of the study interventions varied slightly. For example, coloured signals were used to inform students about their likelihood of success and teachers could also act on the signals to support students in one study. In another study, teachers worked with students to create a proactive plan based on an individual assessment.
The findings from the studies on student grades were quite consistent: all but one study found increasing overall grades and the number of students achieving better grades. The results on retention rates were more mixed with three studies that found higher retention and course completion rates and one study that found that students were more likely to withdraw from the course after receiving the intervention.
The authors pointed out that even though learning analytics can help to accurately identify at-risk students in a rather straightforward manner based on statistical analysis of large sets of data regarding student behaviour, characteristics, and past performance, it is more challenging to use learning analytics to support at-risk students given that the mechanism underlying the effectiveness of the intervention is largely speculative in the studies. Therefore, the question on what is the best way to support at-risk students once they have been identified through learning analytics remains unanswered. Another point made by the authors was that the studies seemed to have place the responsibility of behaviour change on students by giving students feedback on their current learning state and expecting them to change their behavior to achieve better grades. Instead of relying on only students for behaviour change, it might be more effective if both students and the institution/ teachers act on risks and issues identified by learning analytics together. Given that only a small number of studies on learning analytics interventions are published, and hence reviewed, the findings of the effectiveness of learning analytics interventions in the review is limited. Therefore, the authors urge more studies to examine the implementation and evaluation of learning analytics interventions.