We must rethink league tables in light of the Covid crisis

League tables are here to stay, but a different approach could reduce volatility and have long-term benefits for schoo leaders too, write John Jerrim and Loic Menzies

Unions and head teachers have said it would be unfair to compare schools’ performance next year, given the uneven impact of lockdown. They are right. There is clear evidence that there have been considerable disparities in education under Covid and that disadvantaged pupils have been systematically disadvantaged. However, whilst next year’s results will be more volatile and uncertain than ever, year-to-year variation unfairly distorts school performance measures every year, not just when covid strikes.

Fortunately, there is a simple way of reducing the distorting effects of changing cohorts and context. At present, only the last year’s cohort ever ‘counts’ – all previous data is simply binned when it comes to building the headline measure of school accountability, even though longer term measures are available if you dig deeper into the DfE’s datasets. A more rounded headline measure would reduce volatility by drawing in data from previous years.

The DfE knows that this approach would have advantages, particularly when numbers of pupils are small. Its own 2013 report on Reforming the Accountability System for Secondary Schools pledged to introduce three year rolling averages when measuring disadvantaged pupils’ attainment. The DfE should therefore recognise that now is a good time to shift to smarter, fairer accountability.

League Tables are therefore unlikely to be abolished any time soon

On the 30^th of April, with support from Pearson and UCL, we brought together a roundtable of leading academics, alongside officials from the DfE and Ofsted to discuss reshaping headline measures of school performance so that they draw on more than a single year’s data. Today we are publishing a short discussion paper outlining our evolving thinking.

The rationale is this: there are a plethora of difficulties associated with having a single headline measure of school performance and there is considerable evidence to show that current measures tell you as much about a school’s cohort as its actual performance. People also question whether league tables are useful for parents when making decisions about schools (and whether school choice is even desirable or feasible). However, the government is committed to providing parents with a simple measure they can use to exercise ‘school choice’. League Tables are therefore unlikely to be abolished any time soon, particularly given that research by Simon Burgess and Becky Allen shows that “using performance tables is strongly better than choosing [a school] at random”.

What could change though, is how headline measures are constructed.

The big advantage of using more than one year’s data is that you can adjust the weighting of different years’ results – a bit like how results in some subjects (like English and Maths) are weighted differently or double counted. What that means is that the government can stick to its commitments by ‘zero weighting’ grades awarded in 2020, whilst ‘down-weighting’ the potentially volatile results in 2021. Adding in information from 2019 would help provide a more nuanced view and avoid unfairly punishing schools for ongoing disruption.

Of course, constructing any single measure involves trade-offs and one of the things we will be consulting with the sector on as part of our next roundtable is whether old results will feel like an albatross around the neck of fast improving schools. However we believe that in normal years, ‘up-weighting’ the most recent year would provide a counterbalance to this, whilst also insulating schools from being unfairly damned for meaningless statistical noise and changes to their cohort.

The good news is that our recent roundtable provided an opportunity to stress test our methodology, and it seems to be entirely feasible. Our proposed approach therefore offers a short term opportunity to overcome some of the challenges precipitated by Covid, alongside a long term opportunity to end the hire-and-fire culture that comes with judging school effectiveness using a single year’s data.

We are therefore calling on the government to start working towards multi-year measures of school achievement now, so that we can end the untenable game of Russian roulette that teachers and school leaders face each year on results day.