News

Ex-Ofqual chair: ‘Gross miscalculation’ caused 2020 exams fiasco



Last year’s exams fiasco resulted from a “gross miscalculation about what was a reasonable and acceptable way to treat people”, Ofqual’s former chair has said, as he admitted the regulator knew “that in many cases people would be given the wrong grade”.

Of course we knew that it wouldn’t be wholly accurate and we knew that in many cases people would be given the wrong grade

Speaking publicly for the first time since leaving Ofqual in December, Roger Taylor revealed more about what went wrong last summer, when the government was forced to abandon its moderation algorithm at the eleventh hour and issue centre assessed grades.

Prime minister Boris Johnson blamed the chaos on a “mutant algorithm”, but Taylor wrote in a report this week that “the mistakes were made by humans, not machines”.

Speaking at a Centre for Progressive Policy event today, Taylor said blaming the algorithm was the “wrong response” as the problems lay with the policy choices and a “colossal misjudgement”.

“It was in effect a gross miscalculation about what was a reasonable and acceptable way to treat people.”

He added that there was a “risk that we don’t take the right lessons from what happened and if we make that mistake we will continue to repeat the same errors”.

Ofqual
Roger Taylor

‘We knew that it wouldn’t be wholly accurate’

Speaking about the mechanism put in place to award grades last year, Taylor admitted that “of course we knew that it wouldn’t be wholly accurate and we knew that in many cases people would be given the wrong grade, that they would have done better in exam and they would know that, and they would feel a deep sense of injustice”.

It wasn’t that people were blind to it, there was a deep sense of unease as this process was going forward and a realisation that this was an enormous thing to ask of people that they simply accept that this is how it has to be because there’s a pandemic on.

“But somehow that did not sufficiently crystallise in the policy making and the decision making to recognise that this wasn’t just going to wash with people.”

Decisions reflect ‘broader frame of mind’

He said this was “indicative of a broader frame of mind” that wasn’t malicious, but reflected the way that “administrative and bureaucratic processes think about information systems and about data”.

“It’s a frame of mind that is weak at recognising the experience of the individual and thinking about it from the citizen’s perspective, which greatly overvalues benefits in terms of the smooth running of the administration, keeping things on track, enabling the systems to work as closely as they can as they normally work.”

Taylor’s report reveals again how Ofqual had put forward two options last year – to hold socially-distanced exams or use “non-qualification” leaving certificates.

But he claimed the government’s view was that neither approach would “command public confidence”, adding: “I think that view is most likely right.”

“For Ofqual to set itself in opposition to the government in an argument over what would command public confidence and about a policy which stakeholders supported would have been foolish.”

Taylor also said the reason “obviously wrong” grades were not fixed before results day was that Ofqual had “very strong legal advice” that to make changes at that stage would “quite likely result in the whole approach being rejected by the courts following one of the many judicial reviews that a number of law firms planned to request”.



Your thoughts

Leave a Reply

Your email address will not be published. Required fields are marked *

One comment

  1. Huy Duong

    Was it really naivety or was it authoritarianism and arrogance that led to the DfE and Ofqual to think that they could impose so many wrong grades on students?

    https://www.hepi.ac.uk/2020/08/18/how-bad-were-ofquals-grades-by-huy-duong/

    I tried to warn them for months, eg,

    https://committees.parliament.uk/writtenevidence/8239/html/

    “Data from Matthew Arnold School in Oxford (MAS) suggests that for A-levels the Exceptional Arrangements as published so far has virtually no chance of providing grades to the students in a way that satisfies the double criteria of being fair to the individuals and controlling grade inflation nationally. This problem affects every A-level subject at MAS, and is likely to affect most A-level subjects at hundreds of comparable schools across the country. The risk to the students is that fairness to the individuals might be sacrificed.”

    I wrote to them directly many times, after commenting on Ofqual’s blog,

    https://ofqual.blog.gov.uk/2020/04/09/arrangements-for-summer-2020/#comment-3549

    https://ofqual.blog.gov.uk/2020/04/09/arrangements-for-summer-2020/#comment-3550

    https://ofqual.blog.gov.uk/2020/04/09/arrangements-for-summer-2020/#comment-3685

    https://ofqual.blog.gov.uk/2020/04/09/arrangements-for-summer-2020/#comment-3728

    But they were always evasive along the line of “We are refining the details of our algorithm”.

    They would not disclose to the public how their algorithm would ruthlessly go down to really small cohorts, where their use of statistics were invalid, to downgrade 39% of the CAGs and “achieve” 2% grade inflation.

    If they had disclosed that earlier on, they would never have got what Roger Taylor calls “broad consensus”. They knew where they wanted to go, and they kept the public in the dark until they nearly got there.