Evaluating Online Learning: Essential Service or Overhyped?

Search on EES

Author: Gregory McGann

Teaching is difficult.  A 1984 study found that, among students in a class, on average a third learnt the content of the lesson, a third didn’t and a final third already knew the content before it began.  Even among the third who actually benefitted from the lesson, half had forgotten it within six weeks.  With such uncertain outcomes, what hope can we have now that large swathes of world education have been suspended and many bankrupted by the global pandemic? This World Telecommunication and Information Society Day, we examine whether the promise of online education can live up to the challenges it faces. 

The special zeal of the pioneers of free online learning derives from a widespread view that traditional forms of education are failing learners, especially in the poorer parts of the world.  While in many places, including the least developed nations, attendance has rocketed towards the goal of universal primary schooling, attainment in schools has actually continued to lag far behind.  The causes of this “crisis of attainment” are manifold.  For one, student and teacher absenteeism are widespread:  across developing countries, between 14% to 50% of children are absent at any one time.  Attainment fares badly even among regular attendees. In India, which is not untypical of developing countries in this respect, around a third to a half of children at the beginning of secondary school were incapable of reading a simple paragraph.  Across the developing countries, teachers are absent approximately 20% of the time.  Taking into account activities other than what they should be doing, only half of Indian teachers are teaching when they should be.

The situation is even more stark for those traditionally excluded from education, especially the poor and disabled.  Around one in 7 people globally has a disability and nearly 100 million children are estimated to be moderately or severely disabled. In the context of the general crisis in educational attainment in developing countries, the situation with disabled students is even more acute.  In some countries the differential in attendance between disabled and non-disabled students has ballooned: in Indonesia, the difference in primary school attendance is 60%.  Approximately 40% of children with disabilities In low and low-middle income countries are out of school at the primary level and 55% at lower secondary level.  School completion rates for those with disabilities lag far behind those without and seem to be dropping in many developing countries.  Research suggests that this failure may have resulted in a literacy rate for adults with disabilities of only 3% in some countries, dropping to 1% for women with disabilities.  The ambition of “universal design” – an attitude to architecture, technology, education and many other fields that aims to maximise inclusivity – is a long way from being realised in any national or sub-national education system.  Even among those who do manage to achieve success in developing world education, supply is struggling to keep up with demand: by 2025, a projected 200 million places in higher education will be needed.

Enter the revolution in online learning: low-cost and massively scalable.  Shortly after their inception in 2008, Massive Open Online Courses (MOOCs) began to dominate the landscape of online learning with their promise of free, modular education accessible to anyone.  When any of the billion people in the world living on less than a dollar a day can access Harvard’s introductory classes on Computer Science, it is hard to dispute MOOCs’ claim to be democratising learning and their potential as the key vehicle of reskilling has been widely touted by its principal purveyors.

A major obstacle to e-learning, namely access to the internet and internet-connected devices, would appear to have been overcome in many countries.  Almost 60% of households had Internet access at home in 2018, up from less than 20% in 2005 and ¾  of the world’s population owned a mobile phone in 2017.  Some scepticism has been expressed about these numbers but overall the trend is unmistakably one of relentless advance at unprecedented speed. However, headline connectivity statistics cannot be conflated with real “digital engagement” of the kind online education requires.  A frequently interrupted connection, by electricity blackouts for example, sits uneasily with deadlines and timed examinations. Additionally, more subtle barriers often discourage learners: global education communities frequently suffer from important, if often undetectable, cultural clashes that discourage and demoralise some learners.  The dominance of the English language and the almost complete ubiquity of languages of the global north make for another barrier to learners from the global south.

More concerning for those that touted the capacity of e-learning to democratise education are the statistics that show the opposite effect. Research by Coursera, one of the largest MOOC providers in the world, demonstrates that 85% of MOOC participants already have university degrees (although research from fellow MOOC provider edX demonstrates that only a fifth of US graduates regard their undergraduate degree as relevant to their work.)  MOOC completion rates languish at low levels everywhere in the world, but completion, as opposed to enrolment, is even more disproportionately focused on degree-holders in advanced economies.  It has been suggested that a certain attitude,  to both learning and technology, which is gained in higher education, is required for successful online learning.  That suggests that take-up and completion of e-learning programs will not spread suddenly but will follow traditional metrics of progress in education and general acclimatization to a more digital society.


Figure 1: Enrolment and retention in MIT and Harvard edX MOOcs. (The MOOC pivot, Justin Reich, José A. Ruipérez-Valiente, Science  11 Jan 2019: Vol. 363, Issue 6423)


Among the most comprehensive evaluations of online learning was undertaken at Massachusetts Institute of Technology (MIT) – one of the founding institutions of edX – and its conclusions make sobering reading for MOOC advocates.  The multi-year study determines that the growth in online education, particularly MOOCs, has been concentrated almost entirely in the world’s most affluent countries.  Furthermore, the retention rate among learners has remained very low:  in other words, among those who enrol, very few are still engaged on the platform a year later.  Perhaps most concerning, the perennial problem of  MOOCs – low completion rates – is still unresolved.  They conclude that completion rates have not improved in six years.  Quite besides the fact that if completion rates languish at a low level then MOOCs are financially imperilled, the unwillingness to either complete a course or move on to the next is a red flag in pedagogy. Much recent research has focussed on motivation and sustained engagement as key outcomes of any educational intervention because sufficient motivation creates a momentum and an enthusiasm that has been linked with long-term learning success.

In theory, online learning should be well-suited to resolving its own problems: unlike the physical classroom, ample data can be collected efficiently on every learner, bringing the dream of evidence-centred design closer to reality.  With highly detailed data on learner responses, timing and queries, it is possible to imagine Bayesian inference networks updating models of classes and students and proceeding to apply constructed responses to that information.  Unfortunately, that possibility remains remote and, as the world is plunged into reliance on online learning architectures, we are forced to confront their limitations.