A long-awaited Ofsted report on the “reliability” of inspections – tested through sending two inspectors to a school to see if they give the same judgment – will finally be delivered this month.
Joanna Hall, director for schools at the inspectorate, told Schools Week a report on the results of its “double inspections” would be published “this side of March”.
Since April last year Schools Week has been pushing to see Ofsted’s interim evaluation of the inspections, first revealed by the paper in January 2015.
Thirty-five schools were double-inspected, although 60 were “approached”, said Hall.
Double inspections were introduced to demonstrate the objectivity of individual inspectors, following Ofsted’s admission that not enough had been done in the past to ensure the reliability of judgments. They were carried out only in schools previously rated “good”.
About 90 per cent of these inspections resulted in the two inspectors independently coming to the same judgment, Hall told delegates at the Headteachers’ Roundtable Summit in London last week.
But Becky Allen, the director of Education Datalab, said she was “surprised” 10 per cent of inspectors came to different conclusions given they “went on the same day” with the same “bias”.
The assumption is the school is good, so there is the same anchoring bias in each inspector before they go
“The assumption is the school is good, so there is the same anchoring bias in each inspector before they go.
“The fact they’re even making different judgments surprises me, given they’re there on the same day, the same moment in time.”
In a largely critical speech about Ofsted’s use of data, Allen told summit delegates that “consistency of judgment” was not the same as an accurate judgment.
“Making the same judgment doesn’t mean that the whole system is valid. We can consistently make invalid judgments.”
Her criticisms included the inspectorate nearly always rating affluent schools as good while disadvantaged schools were more likely to be graded less than good, according to data.
Education Datalab analysis also showed that Ofsted judgments could not predict whether a school was about to decline or improve.
“Do Ofsted currently manage to identify schools on the cusp of deterioration? No, they don’t. And they also don’t spot those schools about to go on an upward trajectory.”
The system was “not dysfunctional” but with such “short inspections” and an increasing reliance on data, Ofsted judgments should not have the powerful impact they currently have, said Allen.
“We have to lower the stakes. We have to recognise that we don’t want forced academisation and a human being fired on the basis of wrong data and a few hours of another human being walking around a school.”
Schools Week made a Freedom of Information (FOI) request to Ofsted in April last year asking for the report that evaluated the double inspections. Ofsted had said in 2015 that it would publish the evaluation.
It said it had an “interim summary evaluation”, but it could not be disclosed as publication might damage the effects of the on-going study.
The FOI request followed concerns that inspectors in the double inspections checked their judgments were the same before submitting them.
Schools Week then learnt that the interim report would not be published, but a “final” report would be disclosed instead. This is the report that Hall said will be published this month.
“We have to recognise that we don’t want forced academisation and a human being fired on the basis of wrong data and a few hours of another human being walking around a school.” Do we not? I thought that was government policy.