The head of England’s exams regulator has been criticised over his “embarrassing” claim that teacher-assessment grades were “likely to give a much more accurate” reflection of students’ capabilities than exams.
We can feel satisfied it’s likely to give a much more accurate and substantial reflection of what their students are capable of achieving
One former policy adviser even suggested the comment might break Ofqual’s own legal duty to maintain standards in qualifications. The watchdog has since clarified that Lebus was only speaking about this year.
A-level students across England will receive their teacher assessment grades tomorrow, following the cancellation of exams for the second year in a row. GCSE results will be issued on Thursday.
Asked about fears of grade inflation and the validity of the results, Simon Lebus, interim chief regulator at Ofqual, told the Today programme exams were “a bit like a snapshot, a photograph you capture, an instant, it’s a form of sampling”.
Teacher assessments, on the other hand, allowed staff to “observe student performance over a much longer period and in a rather more complex way, taking into account lots of different pieces of work and arriving at a holistic judgement”, Lebus said.
“And I think from that point of view we can feel satisfied it’s likely to give a much more accurate and substantial reflection of what their students are capable of achieving.”
Ex-DfE aides question teacher grades assertion
Several high-profile education figures questioned his assertion. Ministers have repeatedly insisted that exams are the fairest way of assessing children, with plans already in motion to bring them back in 2022.
Sam Freedman, a former DfE policy adviser, said it was a “deeply embarrassing thing for someone associated with assessment to say. I’d argue it breaks Ofqual’s legal duty to maintain standards.”
Ofqual has a statutory duty guide to ensure it maintains “standards and public confidence” in regulated qualifications.
Stuart Lock, chief executive of the Advantage Schools academy trust, added: “I presume he was trying to give this year’s grades some credibility, and he would be right to send that message. But you can’t do it by saying something that is wrong and you know it’s wrong.
“Mainly, it kicks the can down the road for someone else – presumably exam boards, school leaders, teachers, [incoming chief regulator] Jo Saxton – to resurrect the credibility of examinations.”
Ofqual moves to clarify comments
An Ofqual spokesperson said Lebus “was talking in the context of the pandemic and where we find ourselves this year”, and that he was “on the record many times as saying exams are our preferred method of assessment in a normal year”.
“Exams are indeed a snapshot, a sample, in effect, but in a normal year they are taken at the same time by students who have been taught the same curriculum and that’s why in normal times, they are fairest.
“This year students haven’t been able to cover the whole curriculum because of the different course the pandemic has taken. So in the circumstances, using holistic teacher judgements was the most equitable approach.”
But Tom Richmond, another former DfE adviser, said the claim was still “an astonishing statement about this year’s grades as much as any other”.
There is mixed evidence on the accuracy of grades assessed or predicted by teachers.
Research published in 2016 by the Universities and Colleges Admissions Service (UCAS) found that just 25.5 per cent of applicants met or exceeded their predicted grade in 2015.
There have also been warnings that teacher assessment can result in bias against certain groups of students.
‘The bar is very low’
But academics from King’s College London and Queen Mary University of London argued last year that teacher assessments during compulsory education were “as reliable and stable as standardised exam scores”, pointing to research from 2019.
A study last year by Data Educator found the vast majority of predicted GCSE grades were within one grade of being correct.
Dennis Sherwood, an exams expert and vocal Ofqual critic, said Lebus’s assertion “could well be true”, adding that “the bar is very low”.
He pointed to comments from Dame Glenys Stacey, Lebus’s predecessor in the role, who told the education committee last September that exam grades are “reliable to one grade either way”.
Sherwood said it was therefore “not hard for something else – like a conscientious teacher’s holistic assessment – to be better”.
Mr Lebus was right. Exams have a significant error factor both at the margin of all grades and in their reliance on norm-referencing. I fronted a BBC1 television programme in the 1980s exposing their unreliability arguing for change. If anything the situation now is worse. A system of Teacher Assessment externally moderated backed by a school’s ‘licence to assess’ – liable to suspension if light sampling showed they were unreliable – would be much better. Every school would have ‘lead assessors’ for each subject and a Chartered Assessor who would be the expert in each school. Such a system would be cheaper (since it would remove the profits which the three exam boards make form our schools each year) and more reliable. As we reform our schooling system to take account of developments in AI and the need of our future citizens to be able to judge the reliability of information rather than simply its recall, we shall need to change both what we assess and how we assess it. Teachers will be central to that and the sooner we recognise that as other nations already have the better.
Sam Freedman is correct in stating that Ofqual has a statutory duty to “maintain standards and pubic confidence”.
Ofqual also has a statutory duty to “secure that regulated qualifications give a reliable indication of knowledge, skills and understanding…”.
Reliable.
But the legislation does not specify what level of reliability is required. Ofqual have acknowledged that “gold standard” exam grades are “reliable to one grade either way” – implying that an A level certificate showing AAB really means ‘any set of grades from A*A*A to BBC, but no one knows which”.
The BIG QUESTION is “are grades reliable to one grade either way reliable enough?”.
I, for one, don’t think so.
Maintaining standards and properly rewarding pupils for their achievement are incompatible. The only way to maintain standards from year to year is to ration grades. The original idea of GCSEs advocated by Sir Keith Joseph, the architect of the exams (not Kenneth Baker, as popular myth now has it), was that pupils would be assessed according to what they knew, understood and could do as set down by exam criteria. This would be irrespective of the performance of their peers. If a pupil met the prescribed grade criteria they would be awarded that grade.
The answer, surely, is to end this obsession with exams at 16+ and move towards graduation at 18 via multiple routes.
First, Ofqual’s hand-waving “clarification” doesn’t really explain why Mr’s Lebus’ statement that “[exams are] a bit like a snapshot, a photograph you capture, an instant, it’s a form of sampling…[Teacher assessments, on the other hand, allowed staff to] observe student performance over a much longer period and in a rather more complex way, taking into account lots of different pieces of work and arriving at a holistic judgement…” doesn’t apply to normal years.
Second, the bar of grades by exams being only accurate to one grade either way is so low that last year Ofqual testified to the Education Select Committee that it “took comfort” in the fact that its algorithm was mostly accurate to one grade either way, and now it is proclaiming that this year’s grades are much more accurate than grade by exams. Is the bar is so low that a rejected simplistic algorithm is about as good, and what teachers can do with hardly any support from Ofqual is much better?
Third, Mr Lebus conveniently forgot that Ofqual we blindly charging toward 2021 exams, insisting that exams were best, until the PM blindsided them and cancelled exams in January. If teachers’ assessments are much more accurate, why didn’t Ofqual prepare the school and exam boards for it earlier?