Revealed: Ofqual warned of algorithm legal risk

Ofqual warned the government that exam boards could face legal challenges over the issuing of statistically moderated grades, newly published Covid-19 planning documents have revealed. 

A contingency planning document sent to the government on March 16 sets out numerous risks associated with using the controversial approach which ignited last month’s exams grading fiasco . 

The course of action was one of three options proposed by Ofqual who said it could be triggered if “the level of disruption is such that large numbers of students are unable to take exams”. 

Other options included offering additional papers for pupils who miss exams and delaying the start of the exam timetable. 

When listing disadvantages associated with the system Ofqual warned “while it may be possible to achieve an overall statistical fit between the estimated grade, rank order and other evidence, individual students may feel that their final grade is not that which they should be awarded.

“Ofqual and exam boards may be subject to legal challenge over this approach.”

It also stated: “It will be challenging, if not impossible, to attempt to moderate estimates in a way that is fair for all this year’s students.”

The document further predicted teacher estimates “will inevitably include some conscious or unconscious bias” and “will not be able to take account of students who, had they taken their exams, would have done much worse, or much better, than their teachers predicted.”

Ofqual also warns “evidence suggests that particular social and ethnic groups are unfairly disadvantaged in estimates”.

During A-level results day in August it was discovered poorer pupils were more likely to have had their centre-assessed A-level grades lowered during the standardisation process.

Just four days later the government announced both A-level and GCSEs students would be awarded their teacher-assessment grades before standardisation in a major U-turn. 

Elsewhere the planning document said the “credibility of any estimated grades may be questioned, particularly in cases where students’ grades are much worse (or better) than expected.”

Ofqual listed just one advantage for the option, recognising it is “far from ideal” but that it would allows students to have grades “which could be used to determine progression to the next stage of education or employment, and therefore mitigates the risk that substantial numbers of students receive no qualifications, which would put them at a disadvantage in future.”

The watchdog states “in order to make sure the data is collected and checked for completeness , exam boards will need to send instructions to schools by 25 March”.

In practice schools did not receive this guidance until more than a week after this deadline – on April 3. 

Teacher certificates would ‘call into question future of GCSEs’

 

Another option considered by Ofqual was to issue a leaving certificate detailing teachers’ estimates of grades.

The certificates would include a brief commentary and would allow pupils to progress to further or higher education.

However Ofqual said the arguments against this were that it would “call into question the future of GCSEs” and “schools would expect a refund on exam fees”.

Furthermore the document showed how the exams watchdog was looking at prioritising different subjects.

Under the system, vocational qualifications were given the least priority as Ofqual said they are “generally taken alongside GCSEs, rather than instead of GCSEs”.

As they already have “a substantial proportion of centre-assessed work” and “students
may also have already taken some examined units” it was determined there would be “sufficient evidence available to estimate grades if necessary”.

Conversely Ofqual warned that A-levels needed the highest priority as they were used primarily for entry to higher education and “all subjects are therefore equally important for those students using them to progress”.