The decision to use centre assessed grades means many schools are now suddenly in an unfair position and coming under attack from parents and students. Why?
Because they were the schools which heeded the call from education secretary Gavin Williamson to ensure that distribution of grades followed a similar pattern to that in other years, i.e. following the calculated grades model.
These schools are now in an unfair position in comparison with others, which instead focussed on individual pupil grades – leading to those youngsters to get higher grades.
In the interests of fairness to the students, centres whose CAGs were in line with the calculated grades should be allowed to re-submit their CAGs without having to include unrepresentative previous years’ performance in the calculation.
This will not affect large numbers, and will have a very small impact on overall national grades, especially when compared with the big jumps seen in the national figures.
But how did this arise?
There were two incompatible objectives set from the beginning.
On March 20th, the Department for Education said the objective was to ensure students are “awarded a grade which fairly reflects the work that they have put in”. Prime minister Boris Johnson also said the government “would ensure that pupils due to sit exams this summer would get the qualifications they need”.
But at the same time in the same briefing, the DfE also specified a second objective– to “aim to ensure that the distribution of grades follows a similar pattern to that in other years”, AKA comparable outcomes.
Why are these incompatible?
imagine you are the teacher of a class of 30 pupils and you are 75% confident for each pupil that they will get a grade 4 or above, so in line with objective 1, you give them all (100%) a CAG of 4 or above.
But, in practice, if the exams had gone ahead, then there is always variation in exam performance on the day, so typically 75% of them would actually have gained a grade 4 or above , and 25% would not.
And under objective 2, only 75% of the class should get a CAG of 4 or above.
But Ofqual continued this incompatible messaging with their consultation and announcements, as did the exam boards. For instance, the head of centre declaration fit objective 1, and details of the calculated grades fit objective 2.
It is ultimately fair to the whole cohort to follow objective 2. At ASCL (I advise on data) we issued guidance of how to do this: assign an objective mark using assessment data, and that will give rankings.
Calculate a grade distribution in line with previous years and national transition matrices, and assign grades accordingly, making fine adjustments at grade boundaries.
ASCL called this “constrained CAGs” i.e. constrained so as to be non-inflationary overall, but using centres’ fine-tuning (to deal with clustering of candidates).
In June, Datalab also revealed the extent of over-estimation of CAGs – showing they were in some subjects up to a grade more generous. What we saw at the national level were that schools had divided into two clusters – those responding to objective 1 (about two-thirds) and those following objective 2 (about one-third).
Realising that there was likely to be significant over-estimation, Ofqual developed a model so that schools following objective 2 would have the fine-detail of the grading of their CAGs (e.g. where there was clustering so that very similar candidates received the same grade) carried into the final grades, whilst this would not be possible where there was significant over-estimation. This is described in detail in Annex M of Ofqual Interim Technical Report (13th April).
But when the CAGs arrived at the end of June, this model had to be abandoned because of the degree of over-estimation, which for very technical reasons, meant that the objective 2 schools would actually have been disadvantaged.
Because the exam boards needed to have final details of the model in early July, there was not time to try any further refinements. This is explained on pp 308-9 in the Report
So what needs to happen now?
Ofqual and the Department for Education must allow those objective 2 schools to appeal where their CAGs were not representative of previous years.
Schools are in a position to do it as soon as the green light is given, so that students can get the fair grades. They will be able to show evidence because they will have done calculations to use previous years data, and will be able to resubmit with updated calculations based on a fair set of previous data. This should be a straightforward re-running of earlier calculations, and will ensure that this is a robust process