Ofsted is to deploy senior inspectors to “shadow” inspection teams in a bid to test the consistency of judgments under its new report cards system.
The watchdog has confirmed it will introduce extra quality assurance (QA) visits focused specifically on consistency, validity and reliability when inspections resume in November.
It follows concerns that the introduction of more inspection areas and grades will make judgments less reliable. Academics have already expressed concerns about reliability.
We want to know how consistent our inspections are, through testing
Under existing QA checks, senior inspectors shadow less experienced team members during school visits to observe them working, provide guidance and report back to line managers.
But Ofsted plans to build on this by carrying out more shadow inspections to “assess consistency” and ensure grades are “as valid and as reliable as possible”. The watchdog has not said how many more visits it will carry out each year.
Writing for Schools Week, Rory Gribbell, Ofsted’s strategy director, and Dr Verena Braehler, its research and evaluation director, said: “We also want to know how consistent our inspections are, through testing.”
Critics have welcomed the move as a “good starting point”, but have called for regular, “transparent” reporting of the results. They have also called for oversight from an independent, external body.
What is changing?
The new measures essentially mean a ramping-up of Ofsted’s QA process.
“The senior inspector’s role will be specifically to ensure the consistency of inspection outcomes as part of a larger process,” said Gribbell and Braehler.
“After each inspection, any initial differences between senior inspectors and inspection teams will be analysed by our research and evaluation team.”
Feedback will then be considered alongside “wider consistency activity”, which will include inspectors being given simulations of real-world inspections to evaluate their training and judgments.
Inspection toolkits or training could then be tweaked.
Will shadow visits really show inconsistencies?
Gribbell and Braehler said that during QA visits, senior inspectors will “advise and guide the inspection team to the right result” before reporting back to Ofsted about areas where they reached different conclusions.
But Dr Tim Leunig, a professor at the London School of Economics and former DfE policy adviser, is concerned junior inspectors will simply “follow” the senior inspector’s judgment.
He said a fairer measure would be if junior and senior inspectors filed separate judgments “without having seen the other verdict, and both are revealed together”.
Transparency is key
University College London academics John Jerrim and Dr Sam Sims, and The University of Southampton’s Professor Christian Bokhove, have long argued for greater scrutiny of inspection reliability.
They described Ofsted’s move to carry out more shadow inspections as “a real positive”.
But while “a good starting point”, they warned the watchdog was setting “a low bar”.
They fear the process is “likely to only show up really quite major instances of inconsistencies”, analogous to “when two referees disagree on whether a player is standing three yards offside”.
More work would be needed to prove inspections had a high degree of consistency and reliability.
Crucially, they said “results of research on consistency [must be] transparently reported”.
They also called for a “close external overview” of this work, “ideally…conducted by an independent organisation”.
Dearth of reliability evaluations
But Ofsted has “to start somewhere”, they said, adding that “for 30 years we have had pretty much nothing else”.
The last evaluation of Ofsted grade reliability was published in 2017, before the latest inspection framework was introduced.
It found two inspectors tended to agree on which grade to award a school, but the report only looked at “short” inspections of schools already rated ‘good’ or better.
Other studies have shown wider discrepancies. One by Bokhove, Jerrim and Sims in 2023 found that primary schools assigned a female lead inspector were around one third more likely to receive an ‘inadequate’ judgment.
‘Not a one-off’
Jerrim and Bokhove stressed Ofsted’s new work on consistency must not be “one and done” and “needs to be a much longer-term endeavour.”
Ofsted has said results of the new consistency work will be published next year.
It is not clear exactly when, but the watchdog said it was keen to make any necessary updates to the inspection framework ahead of the 2026-27 academic year.
But Gribbell and Braehler insisted the work not be a “one-off exercise”, and that regular reports will be published.
Your thoughts