Two-thirds of Ofsted grades now match results after pandemic dip

Around two-thirds of school inspection grades now match performance outcomes, Ofsted analysis shows, after a dip in alignment during the pandemic.

New research from the watchdog comes after Schools Week reported on the broken link between exam results and Ofsted ratings while inspectors were results blind during Covid.

Incoming chief inspector Sir Martyn Oliver has also raised concerns over “consistency” between inspections and outcomes, while former HMCI Sir Michael Wilshaw has warned the inspectorate has “moved too far away” from using data in inspections.

The analysis shows that of inspections that took place in 2022-23, 68 per cent of primary and 64 per cent of secondary school results corresponded to overall effectiveness grades, based on either key stage 2 or progress 8 scores from the previous year.

Results ‘similar to pre-2019’

Ofsted said there was a “similar level of alignment” in 2018-19, before the new framework, which focused more on curriculum and less on results, was introduced.

In 2018-19, 65 per cent of primary inspections and 67 of secondary inspections had Ofsted grades that aligned with outcome data.

Ofsted also found that when looking at 2022-23 performance outcomes in comparison with inspection outcomes for the same year, 68 per cent of primary and 64 per cent of secondary Ofsted grades aligned.

“This suggests that inspectors are identifying the quality of education in schools, even when outcomes data lags behind what is actually happening at the school at the time of the inspection,” the report said.

But Dave Thomson, chief statistician at FFT Education Datalab, said he was “unsure” that the analysis showed a “good degree of alignment” – pointing out 76 per cent of primary schools were judged ‘good’.

“So on the basis of no information at all, you would be able to guess the outcome of a primary school inspection correctly just by saying they were all ‘good’.

“In that context, an alignment rate of 68 per cent doesn’t seem particularly high.”

Ofsted alignment dipped during Covid

Earlier this year, Schools Week found that alignment had dipped after exams were cancelled during Covid, leaving inspectors without access to performance data.

In 2018-19, under the old framework, the correlation “coefficient” between Progress 8 scores and inspection outcomes was 0.59. (1 indicates a perfect link, and 0 indicates no link).

Ofsted’s annual report shows this fell to just 0.46 across the 2021-22 academic year when there were no national results to consider.

We also found a growing trend of “outliers”. Between 2017 and March 2018, just 1 per cent of schools rated ‘good’ had Progress 8 scores of below -0.5.

Between September 2019 and March 2020 – after the introduction of the EIF – the figure grew to 8 per cent, and early data for 2022-23 suggested it had risen again to 10 per cent.

New analysis shows eight schools in the bottom 100 for Progress 8 scores were rated ‘good’ last year, while one school in the top 100 for Progress 8 in the country was rated ‘requires improvement’.

Ofsted: ‘Good reason’ for outliers

Ofsted’s analysis found more than 90 per cent of schools where their grade and results did not align were only one inspection grade out.

Fewer than 100 schools were more than one inspection judgment away from their data band.

The inspectorate also said there were “good reasons” behind the differences.

For instance, outcomes could “lag behind changes” in a school, or weak SEND provision could pull down quality of education judgments, even if other pupils are “generally doing well”.

Above: thresholds used by Ofsted in its analysis.

Schools with higher levels of disadvantage were also more likely to have lower performance data, while inspectors considered outcomes “in context”.

Also, 72 per cent of secondaries graded lower than their data band had a behaviour and attitudes judgment that was also lower. The comparative proportion at primary level was 39 per cent.

There was also a particular issue with small schools, which were more likely to have difference inspection grades to their data band. This was because results were a less reliable indicator of underlying quality.

This “volatility” also applied to a “small number” of secondary schools, which were rated ‘good’ despite having among the lowest results.

However half of these were university technical colleges which “often give a good technical education” which is “not always well reflected” in Progress 8 scores.

‘We need more insight into inspector black box’

In his pre-appointment hearing for the top Ofsted job, Oliver told MPs it was “difficult to explain how some schools had a ‘good’ quality of education and some of the best outcomes in the country and I think it’s difficult to explain how you get some of the worst outcomes in the country getting a ‘good’ inspection”.

In evidence to the committee’s inquiry into Ofsted’s work last month, Wilshaw described what he said was a “ridiculous position” of schools with low progress scores and “terrible outcomes” gaining ‘good’ Ofsted judgments.

John Jerrim, professor of education and social statistics at UCL’s Institute of Education, said the new analysis suggested data was not “playing any less of a role” under the 2019 framework.

But he added: “What we really need is a bit more insight into the black box of how inspectors reach their decisions – including the role played by background data, and how much inspectors views are already formed even before they set foot in the school.”