Abstract

Virtual learning environments (VLEs) form part of modern pedagogy in education, they contain student usage data that has potential to inform and improve this pedagogy. The question this thesis explores is how might the development of data mining and log analysis systems for the Moodle virtual learning environment, improve computer science students’ course engagement and encourage course designers’ future engagement with data analysis methods for the evaluation of course resources? The thesis proposes that a student will complete missed work sooner if their utilisation of the VLE is automatically tracked and electronic prompts are sent when VLE activities are missed. The thesis also hypothesises that presenting a simple-to-use data mining and visualisation tool to course designers would increase their future acceptance of data mining technology for informing course design with a longer-term intent that this would improve the quality of the online learning experience ultimately improving student engagement. Exploring the two hypotheses required the development of two software artefacts, MooTwit – a tool that contacts students when they fall behind in their VLE study and MooLog – a tool that extracts and presents summative information on VLE course utilisation.
To establish if student timely engagement improved, the study used MootTwit with two groups of students over a period of 15 weeks, messaging one group only when they fell behind. Statistical analysis and comparisons were made between how quickly each group engaged with the missed items. Using MooTwit to track and contact students did influence the timeliness of their engagement with the VLE activities. Specifically, the results suggest by direct messaging a student to engage with missed material, they complete missed activities closer to required completion date. To ascertain if the acceptance of data mining for course evaluation could be improved, surveys were used before and after a demonstration of MooLog to a group of course designers. The pre-demonstration survey assessed existing planning and evaluation processes. The post-demonstration survey collected evaluations of the relevance of the information provided by MooLog and the likelihood of
Page 3 of 120
the software being used to evaluate course effectiveness. The results of the study established that many designers currently do not use data analysis as a method of informing course improvement and there was evidence to suggest the MooLog demonstration significantly increased acceptance of the potential of data mining.
The findings within the thesis show that educational data-mining has the potential to improve pedagogy in VLE linked education offering opportunities to increase timely engagement and to raise course designers’ acceptance of data mining to improve the validity and quality of course evaluation.