Creative remote education can make up for learning lost during school disruptions
NOAM ANGRIST is a cofounder of Youth Impact and a fellow at the University of Oxford.
Opinions expressed in articles and other materials are those of the authors; they do not necessarily reflect IMF policy.
The COVID-19 pandemic was a historic shock to education systems. In many low- and middle-income countries, it dramatically set back learning levels.
Even before the pandemic, educators were talking about a global “learning crisis.’’ For example, in Kenya, Tanzania, and Uganda, three-quarters of grade 3 students cannot read a sentence such as “The name of the dog is Puppy,” according to an assessment by Uwezo, a regional initiative to measure education quality.
Enrollment in school has risen to record highs. The average adult had completed 7.6 years of school in 2010, more than double the average of 3.2 years in 1950, based on an analysis of data from 164 countries. In the past decade, primary school enrollment rates in sub-Saharan Africa have risen from 80 percent to 92.3 percent. Yet in many countries, learning levels have not improved much (see Chart 1).
The COVID-19 pandemic has further arrested learning progress. More than 1.6 billion children across 180 countries were out of school at the height of the pandemic. Using past disruptions as a benchmark, the cost of school disruptions is likely to be massive and have long-term consequences. For example, in 2005 an earthquake in Pakistan disrupted schooling for 14 weeks; four years later, young children most affected by the earthquake performed significantly worse on learning assessments, according to research published in the Journal of Human Resources (Andrabi, Daniels, and Das 2021). Some countries, such as Sierra Leone, closed schools during COVID-19 for a similar 14 weeks, but many school closures have been much longer. In Uganda and the Philippines school disruptions lasted nearly a full two years.
While many governments launched ambitious remote learning efforts, such as radio and TV campaigns, emerging evidence suggests that substantial learning was lost during the pandemic. Research in Brazil, India, The Netherlands, and South Africa, for example, revealed learning losses so large it appears that very little was learned during school closures. A few randomized evaluations in Kenya and Sierra Leone have found limited effects of various remote learning interventions.
But not all remote learning was ineffective. In an experiment in Botswana, weekly text messages coupled with phone call tutorials to parents and their primary school children improved learning (see Chart 2). The program covered foundational numeracy concepts and comprised weekly 20-minute tutorials over the course of eight weeks. Results provided some of the first experimental evidence during the pandemic on approaches to mitigating learning loss. Not only did it work, the intervention was also cheap and cost-effective, yielding the equivalent of over one year of high-quality instruction for every $100 spent. Text messages alone were not effective—some level of live, direct instruction via the phone was essential.
The experiment in Botswana showed that phone call tutorials, targeted to students’ level of learning, improved primary school students’ grasp of math concepts while schools were shut down during the pandemic.
One reason the phone call approach to remote learning in Botswana was effective is people’s broad access to mobile phones at low cost. In low- and middle-income countries, 70 to 90 percent of households own at least one mobile phone, while only 15 to 60 percent of households have internet access. Relying on technology that requires internet access may not work in many low- and middle-income settings. Low-tech approaches can reach the most marginalized and do so at scale.
Another reason for the Botswana approach’s success: it customized and targeted instruction to each child’s level rather than relying on a one-size-fits-all curriculum. A weekly problem was posed at the end of each session to assess children’s levels; for example, whether they could do single-digit addition (4+5). If they could, the instructor would move on to more difficult problems, such as subtraction (7–3). If they couldn’t, the instructor would continue teaching addition. This approach built on a large body of literature showing that targeting instruction to a child’s learning level is one of the most cost-effective approaches to improving education outcomes.
To understand why targeting instruction is so effective, consider the status quo. Most education systems are structured by grade and follow a strict grade-level curriculum. For example, children are expected to know two-digit division by grade 5. But in practice, most children do not. Data from Botswana, for example, show that fewer than 10 percent of grade 5 students have mastered two-digit division. Yet teachers often still teach the grade curriculum, and students are promoted to the next grade whether or not they grasp the core concepts. Instruction that prioritizes curriculum over competence, coupled with automatic promotion policies, is common in many low- and middle-income countries: as a result, many children fall behind grade level, and stay behind. In this context, assessing learning levels, regrouping children by level rather than grade, and targeting instruction can be transformational.
A particular model of this approach, called “teaching at the right level,” has been gaining ground in schools across sub-Saharan Africa, as well as in India, where it was pioneered by the education organization Pratham and evaluated by J-PAL, the Abdul Latif Jameel Poverty Action Lab (Banerjee and others 2017). Customizing instruction to children’s learning level might seem challenging, but with a few structures, such as frequent diagnostic testing and a menu of activities for each level, this approach has been adapted for over 60 million children. In Botswana, a coalition of the Ministry of Basic Education, the Ministry of Youth Sports and Culture Development, the US Agency for International Development, UNICEF, Teaching at the Right Level (TaRL) Africa, and Youth Impact, one of the largest nongovernmental organizations (NGOs) in the country, had delivered teaching at the right level to more than 20 percent of primary schools right before the pandemic. (The author is a cofounder and executive director of Youth Impact.)
When the pandemic struck and shuttered schools, Youth Impact pivoted to provide targeted instruction on foundational numeracy using the low-tech mobile phone approach. Both the platform (mobile phones) and pedagogy (targeted instruction focused on foundational numeracy) were crucial for the approach to work. In addition to targeting instruction through weekly assessments, the phone call from instructor to student was conducted one-on-one rather than in a group classroom setting. This one-on-one interaction enabled even more targeted instruction, an innovation that can be carried forward beyond the pandemic. This approach also ties in with a great deal of literature on the striking effectiveness of tutoring. However, the tutoring literature is often focused on high-income settings, and tutoring can be expensive. New, cheaper models have emerged during COVID-19 in Italy, where university student volunteers provided free online tutoring to disadvantaged middle-school students, and Spain, where math teachers offered online tutoring after school hours. The Botswana study provides a model of inexpensive tutoring at scale in low- and middle-income settings.
Since the Botswana study was released, similar approaches have been tested and shown to be effective in Bangladesh and Nepal. Moreover, an ongoing randomized trial across five countries (India, Kenya, Nepal, Philippines, Uganda) is testing the adaptability and scalability of this approach across contexts. For example, the multicountry study includes delivery by NGOs as well as government teachers.
Although the pandemic has stymied the progress of education, and many efforts to provide remote instruction during school closures have failed, those that have worked combine evidence from the past with contextually grounded innovation. The Botswana study is one such example, building on decades of evidence on teaching at the right level and tutoring, while innovating to reach people where they are—which dramatically changed during the pandemic, with children at home and using phones, rather than sitting in a classroom.
A recent review of prior evidence, as well as innovations tested during COVID-19, can guide the way (Angrist and others 2020). The Global Education Evidence Advisory Panel—an independent academic advisory group convened by the World Bank, the UK Foreign, Commonwealth and Development Office, and UNICEF—has done just that in a new report, “Prioritizing Learning during COVID-19: The Most Effective Ways to Keep Children Learning during and Post-Pandemic.”
The report highlights multiple cost-effective approaches to improving learning. The most important one is to keep schools fully open. Other reforms include assessing student learning to guide and track learning progress and to enable teaching at the right level, structured pedagogy, and provision of additional instructional support, such as tutors. Notable lessons learned during the pandemic include leveraging existing technology, such as adaptive software, to target instruction where such infrastructure exists, and where it doesn’t, leveraging high-access mobile-phone-based instruction. Another lesson involves engaging parents directly in instruction. Before the pandemic, parental engagement was focused more on informational interventions, such as report cards. During the pandemic parents became frontline instructors, and emerging evidence suggests that in some cases they were quite effective. This was true especially when interventions focused on foundational skills, enabling parents from low- and middle-literacy settings to engage. A point to consider in designing effective parent support interventions is to keep them brief to enable high engagement and to avoid crowding out employment.
COVID-19 devastated education systems worldwide. While the window to recover learning losses is closing, it is still possible to do so if we act now. But we cannot just go back to business as usual or we will wind up back where we started: with a learning crisis. This is the moment to take stock of what hasn’t worked and what has, and to reform education systems to prioritize and enable learning for all.
Andrabi, Tahir, Benjamin Daniels, and Jishnu Das. 2021. “Human Capital Accumulation and Disasters: Evidence from the Pakistan Earthquake of 2005.” Journal of Human Resources 0520-10887R1.
Angrist, Noam, Peter Bergman, and Moitshepi Matsheng. 2020. “School’s Out: Experimental Evidence on Limiting Learning Loss Using ‘Low-Tech’ in a Pandemic.” NBER Working Paper 28205, National Bureau of Economic Research, Cambridge, MA.
Angrist, Noam, David K. Evans, Deon Filmer, Rachel Glennerster, F. Halsey Rogers, and Shwetlena Sabarwal. 2020. “How to Improve Education Outcomes Most Efficiently? A Comparison of 150 Interventions Using the New Learning-Adjusted Years of Schooling Metric.” World Bank Policy Research Working Paper, Washington, DC.
Angrist, Noam, Simeon Djankov, Pinelopi K. Goldberg, and Harry A. Patrinos. 2021. “Measuring Human Capital Using Global Learning Data.” Nature 592 (7854): 403–08.
Banerjee, Abhijit, Rukmini Banerji, James Berry, Esther Duflo, Harini Kannan, Shobhini Mukerji, Marc Shotland, and Michael Walton. 2017. “From Proof of Concept to Scalable Policies: Challenges and Solutions, with an Application.” Journal of Economic Perspectives 31 (4): 73–102.
Lee, Jong-Wha, and Hanol Lee. 2016. “Human Capital in the Long Run.” Journal of Development Economics 122:147–69
