The ill-fated exams “triple lock” policy that sparked a chain of events culminating in pupils instead being awarded their centre-assessed grades was announced before Ofqual had been able to even sign it off.
The exams regulator has today published its long-awaited board minutes that give a behind-the-scenes account of this year’s exams fiasco. They also show education secretary Gavin Williamson considered issuing pupils their CAGs before A-level results day, and just two days before saying he had confidence in the system.
They also reveal ousted chief regulator Sally Collier rebelled against Williamson’s last-minute triple lock appeals policy, and his replacing her was discussed in advance of her resignation. The exams regulator also admitted its calculated grades were not reliable for some “outlier” pupils, but said they had no way to solve that – apart from allowing those pupils to appeal.
The publication follows Schools Week’s exclusive this morning that Williamson had met regularly with Ofqual before results were published, shooting down his claim of ignorance over the issues.
1. Williamson considered issuing CAGs days BEFORE A-level results
Minutes of an emergency Ofqual board meeting in the afternoon of August 11 state that Williamson had that day raised “options” including the issuing of centre-assessment grades. He had also raised the possibility of allowing individual appeals against calculated grades, or allowing mock exam results to be awarded if they were higher.
The meeting, revealed earlier today by Schools Week, had been convened at short notice to consider options which had been raised earlier that morning by the education secretary – referred to in minutes as the SoS – to address declining public confidence in calculated grades and the events in Scotland.
The findings are also somewhat surprising given that Williamson wrote to Ofqual just two days later, saying he was “confident that the approach which Ofqual has designed means that the vast majority of students will get the grades they deserve this summer”.
The documents note that Williamson “had indicated a preference for a fast track way for students to receive their mock result instead of their calculated grades if the mock result was higher and for the process to be completed by September 7 2020”.
2. ‘Triple lock’ proposal was announced before Ofqual signed off
Minutes of an emergency meeting held on the evening of August 11 show the government announced its “triple lock” plan before Ofqual’s board had agreed to it.
In fact, the announcement was made to the press and widely reported while the meeting to discuss it was still in session.
Board members were read a draft press release on the triple lock policy, which was described by Ofqual at the time as a “new development”.
The minutes also show that Ofqual advised the DfE it had not consulted on students receiving their mock result following appeal and that its board “had not as yet considered or agreed the proposal”, but the DfE advised “that they would proceed with the press statement”.
According to the minutes, the board “took a short break” while Collier spoke to Williamson, who was “concerned that this issue was resolved quickly”.
The board agreed that Ofqual “would need to provide a response to the proposal and work with the SoS to ensure progression for all students whilst not compromising further on standards”, and would need to see the detailed policy position in order to do this.
However, a press release had already been circulated, and the proposal was “widely reported in the media while this meeting was still in session”.
3. Mocks plan ‘diluted and compromised standardisation process’ …
As the press release was read out on the evening of August 11, board members noted “the difficult situation this now caused”.
The wording “suggested a diluted and compromised standardisation process, even given the opportunity for Ofqual to define what would constitute a valid mock”, the minutes state.
The board also noted the “risks associated with the government policy decision which could be in tension with the independent regulatory system”.
Collier told the board her executive team had “tried to fulfil Ofqual’s role in accordance with its objectives whilst being cognisant of the outcomes that the DfE was targeting”.
“However, if Ofqual acceded to this request, this would be a fundamental shift. The chief regulator could not, as the independent regulator, accede to a request that contradicted what Ofqual had been established to do.”
The board also warned over the consistency of mocks, adding while the triple lock would benefit outlier candidates it would “further disadvantage” private candidates.
4. … but ‘risk of collapse’ in Ofqual’s credibility in not implementing already-announced policy
At a meeting on August 14, Collier “highlighted the risk of collapse of public confidence in the examination system, and also to credibility of Ofqual as the regulator of that system were Ofqual not to implement this policy given it had already been announced”.
The board discussed a “key mitigation” of allowing appeals on the basis of non-exam assessment results as well as mock exams, “because this would open the appeal route to students who had been unable to take a mock”.
The board was of the view that following the direction from Williamson “was likely to result in less valid outcomes overall, with unfairness at individual and centre level”, but that this was to be balanced “against the obvious public confidence considerations, given the announcement and the falling away of confidence in recent days”.
The board then resolved to implement the direction to allow an appeal “on the basis of a student having a valid mock or non-exam assessment result which was higher than their calculated grade”.
It also resolved that on a successful appeal, the grade awarded should not exceed the CAG.
5. Ofqual chief rebelled over mocks plan
Ousted chief regulator Sally Collier was vocal in her support to ditch the proposed mock appeal route and instead use CAGs from quite early on, the minutes show.
During an emergency board meeting on Saturday August 15, she disagreed with the board’s decision to withdraw its guidance on using a CAG cap under the mock appeal route (the board later decided to remove the guidance).
During a meeting the following day, Collier also called for the board to just go ahead with using CAGs and to tell Williamson this was their call. But the board instead resolved to reissue the mocks appeal guidance and meet again to discuss whether to use CAGs. Collier voted against this.
6. After causing a fuss over the mocks CAG cap – Williamson then accepted this was the best route forward
A meeting had to be held on the evening of Saturday August 15 as Williamson had asked Ofqual to reconsider the CAG cap applied to the proposed mock appeals route (Ofqual took down its guidance on this and resolved to decide a way forward on Sunday).
But, incredulously, after agreeing to remove the CAG cap during a board meeting at 1pm on Sunday, the board was told at a further meeting at 5pm that Williamson now believed they should go ahead with the mock appeal route – but include the cap (rather than just awarded pupils their CAGs).
The minutes read: “The Board noted that the situation had continued to change, and there were increasing and material risks in the implementation of the ‘mock appeals’ approach. It was concerned that delivery of a mock appeals process was in serious doubt at this late stage. There were clear signals that this was not achievable, even with a suggestion of moving GCSE results day.”
The board heard the mock appeal route “risked stretching exam boards beyond achievable limits” and highlighted a “notable and irretrievable falling away of public confidence in the standardisation approach”.
7. Ofqual knew of ‘volatility’ for outliers, but said ‘no valid’ solution other than appeals
An emergency board meeting on Tuesday 4th was called to discuss appeals guidance. It showed that the “data distribution of a small cohort or entry is particularly subject to volatility, and that apparent inconsistencies between CAGs and historical data could be due to either real differences in student performance or to instability related to the size of the available historical data set”.
“Those causes are indistinguishable when viewing or analysing the data set alone and would require additional information to differentiate. It would require judgement to identify and remedy cases – there would need to a defensible rationale as to whether a candidate’s grade was an outlier or was not, and what the remedy ought to be.”
The board was “very concerned about the prospect of some students, in particular so-called outliers, being awarded unreliable results”. Despite the significant concerns, and knowing some pupils would get unreliable results, the board “accepted reluctantly that there was no valid and defensible way to deal with this preresults, and that it would therefore have to be addressed via the appeals route”.
The problem of outliers getting “less reliable” calculated grades – and the potential for a “significant change in the shape of a centre’s expected grade distribution” – were first highlighted at a board meeting on July 29.
8. Plans for Collier’s resignation were discussed in advance
A meeting was held on August 23, with the only thing on the agenda to discuss “contingency arrangements in the event that Ofqual is without the chief regulator at short notice”.
“The board noted that should the chief regulator resign, then the Board would need to act swiftly to prevent the loss of senior skilled staff, to strengthen the organisation’s capabilities and to rebuild Ofqual’s reputation.”
The rest of the discussion was redacted. The meeting wasn’t attended by the then chief regulator Collier, but was attended by her successor Glenys Stacey. Collier ended up resigning two days later.