News

Ofsted ‘double inspection’ pilot thwarted after inspectors ‘spoke frequently’

Exclusive

Ofsted had to beef up safeguards during its double inspection reliability study after inspectors, who were supposed to independently come to final judgments, were found to be discussing their findings, according to a report released by the inspectorate today.

The education watchdog has this morning published the findings of its tests on the reliability of short inspections.

Schools Week first reported the pilot in February 2015, and had an exclusive preview of the findings.

The tests involved two inspectors visiting 26 primary schools and delivering an independent verdict of whether the school should remain good, or needed a full inspection.

In 22 of the 24 cases analysed (two were dropped from the final analysis) inspectors came to the same conclusion.

Inspectors were doing minor things that could prejudice judgments

However the report reveals an earlier pilot of the reliability study in summer 2015 – involving seven schools – found some inspectors had “spoken frequently” during their inspections.

In two instances, inspectors came up with “single shared outcome”, as opposed to separate decisions.

The report explains how in each case one inspector had identified the need to continue on to a full inspection, then discussed the findings with their colleague, before the pair subsequently reached the same decision.

Schools Week has previously revealed that these findings were understood to have been raised in an interim report of the study – which will not be published.

However the final report, published today, does include a summary of the interim review findings.

Alan Passingham, one of the report’s authors, told Schools Week that inspectors “didn’t quite get what we were trying to achieve” in the earlier pilot.

“They were doing little things, such as a wave to each other – minor things that could prejudice judgments.

“We took that on board and included protocols [in the final study] to ensure less of that kind of behaviour was going on.”

In the second round of reliability tests, inspectors had to complete summary evaluation forms and place them in sealed brown envelopes handed to their boss before meeting to agree a final outcome.

Inspectors were also given different roles, with one required to “concede authority” to another if their inspections overlapped.

Ofsted inspections are unreliable by desig

And independent observers were appointed to oversee some inspections, following discussions with Ofsted’s expert panel of advisors.

Passingham added: “We can’t be absolutely sure we’ve managed to nullify [all problems], but there is enough evidence to reassure us that if there was interaction it was likely to be unintentional.”

The two inspectors did still meet the headteacher at the same time – at the start of the inspection – to develop lines of enquiry.

The report stated this was likely to introduce an “element of confirmation bias” to the study, but added this was done to avoid additional burden on the school.

Save

Save

Save

Save

Save

Your thoughts

Leave a Reply

Your email address will not be published. Required fields are marked *

One comment

  1. James Mook

    1. “We can’t be absolutely sure we’ve managed to nullify [all problems], but there is enough evidence to reassure us that if there was interaction it was likely to be unintentional.”
    The fact that it is unintentional is not the issue The fact that it may unconsciously affect outcomes is.

    2. “The report stated this was likely to introduce an “element of confirmation bias” to the study, but added this was done to avoid additional burden on the school.”
    Additional burden is probably a small price to pay to remove any “element of confirmation bias”

    3. “They were doing little things, such as a wave to each other – minor things that could prejudice judgements. We took that on board and included protocols [in the final study] to ensure less of that kind of behaviour was going on.”
    Good that they are trying to minimise this, but the fact that “less of that kind of behaviour” was happening concerns me. Les of that does not mean none. And since the inspectors go in knowing the school is good, I could imagine it wouldn’t take much of a signal to say “things are staying the same here!”