5 reasons to be cautious about estimates of lockdown learning loss

Research estimating lockdown learning loss, published during the crisis, paints a grim picture of widening attainment gaps and long-term effects.

But how robust are the findings? There are reasons to treat with care the reports put into the public domain so far.

1. Much of the research remains speculative

Covid-19 and the school shutdowns it has led to are unprecedented. There are no easy examples to fall back on in order to know for sure what will happen next.

We do know that there are sharp disparities in children’s experience of lockdown learning. Those stemming from household poverty have rightly raised the most concern.

Children cooped up in a small space with inadequate access to a quiet place to study, no access to technology to support learning online, and insufficient room to play, will have had a very different experience from children with access to all of the above and more.

The Covid crisis has drawn attention to how little some communities have to fall back on and how important schools’ role is in ensuing children are safe and well-fed, can exercise outside, and socialise with their friends, as well as have opportunities to learn in interaction with teachers and their peers.

2. Drawing parallels with other school closure contexts isn’t straightforward

Natural disasters, ranging from long-term damage to schooling caused by Hurricane Katrina, the impact of the Christchurch earthquake in New Zealand, or more intermittent weather-induced closures elsewhere, do provide some data. But it is hard to disentangle the effect of the absence of schooling from the effect of the disaster itself on children’s mental health and well-being and therefore their capacity to learn.

Other disruptive events, such as prolonged strike action or school boycotts, sometimes used as reference points, may be as misleading if the action is provoked by factors that independently shape children’s learning.

3. Using extended absence as a proxy for learning lost is not a precise guide

Children who are frequently absent from school do less well than children who attend regularly. But absence may be a proxy for other difficulties pupils are experiencing. And absent children face the added difficulty of coming back into a class where learning has moved on.

4. Calculating exactly how much learning has been lost is more complicated than it might at first appear

This is not just to do with the absence of good data. During the lockdown, teacher and parent survey data has been used to calculate the % of children who don’t have access to online learning – but is online learning more beneficial than learning using hard-copy resources?

Parents and teachers have also been asked to calculate how much work has been set, and children have done. That equates time on task with learning.

It doesn’t consider the value of the task itself, nor the value of other undocumented activities – say wider reading or household activities – that children undertake. We may exaggerate the losses unfairly.

5. Measuring what has been lost or how easy it will be to regain?

Most research on learning loss measures the gap, typically the difference in test scores before and after the summer holidays in the USA.

But more recent research, reviewing children’s progress over two consecutive years, reveals that learning does not travel in a straight line, and interruptions have unexpected effects (Kuhfeld et al, 2020).

Those gaining most during the first year were likely to lose most over the summer, but then made up those gains thereafter.  Learning both speeds up and slows down with gains and losses spread unevenly over the year – some children gained more over the summer and this could not be fully determined by levels of disadvantage. The same researchers suggest that Covid will lead to more variation in student skills when schools re-open.

What lessons can we draw from this?

That a rush to judgement about how much has been lost is unlikely to be accurate, over either the short or the long term. That schools should expect uneven results from the lockdown for their pupils. They need time to accurately assess how best to rebuild.

That rebuilding is a long-term project, not a short-term quick fix. That schools should be able to draw on additional resources to support those learners that may have been impacted most, but that the form that support takes should rest in the hands of schools to determine and commission.

Funding should recognise those children living in poverty or in communities hit hardest by Covid will have the greatest need.

*This article is based on research undertaken by the ESRC-funded ‘A duty of care and a duty to teach: educational priorities in response to the COVID-19 crisis’.