The key stage 2 reading test was probably “unduly hard” for pupils with low attainment and special educational needs in 2016, according to exams watchdog Ofqual.
The comprehension test, which was part of the “tougher” primary curriculum started in 2014, was heavily criticised on social media after it was much harder than teachers expected.
SATs results in 2016 showed just 66 per cent of pupils met the standard for reading, compared to 70 per cent in maths and 74 per cent in writing.
Only around half of pupils (53 per cent) met standards in all three areas compared to 80 per cent the year before.
Ofqual has now reviewed the approach taken by the government-led Standards and Testing Agency which designed the reading test.
In particular, Ofqual criticised that the body “did not forsee the intensity of concerns” raised by teachers even before the results came out.
Schools Week regularly reported throughout the year on confusion and worries by teachers surrounding the tests – however, the government forged ahead.
In future, Ofqual has recommended the vetting process the STA uses to judge the tests – including a teaching panel, test review group and inclusion group – might need to “run differently” given the concerns were missed.
Ofqual’s report also found “abrupt transitions” in the middle of question sets, from low-demand to high-demand questions, which could have been difficult for many pupils.
Discussions also highlighted the use of “idiomatic” language which could have disadvantaged pupils with special needs or English as an additional language.
An “old-fashioned feel” to the vocabulary could also have been hard for pupils from poorer backgrounds since these words may not be used at home, representatives told Ofqual.
The exams watchdog agreed it “seemed plausible” the reading test had been “unduly hard to access for at least some pupils” but Ofqual stopped short of saying which pupils were affected.
“We do not have sufficient evidence to reach any definitive conclusion concerning which pupils might have been affected in this way, nor how many pupils, nor to what extent,” said Ofqual.
However, social media comments from teachers claiming even high-ability pupils struggled were not backed by the results, said the report.
For our special needs pupils it was almost pencils down
About a quarter of pupils were estimated not to have reached the end of the reading test, but the majority did. One in 20 pupils did not reach the first question for the third text.
Christine Kemp-Hall, executive principal at North Ormesby primary academy in Middlesbrough, part of the Academies Enterprise Trust, said her staff were “horrified” when they had opened the paper.
“For our special needs pupils it was almost pencils down,” she said.
Archaic words such as “ancestors”, or a picture of four kinds of columns asking which was a “monument”, would have challenged some secondary school pupils, let alone primary ones, she said.
Such questions had been too difficult for the school’s poorest and most vulnerable pupils as well, who had struggled alongside special educational needs pupils, said Kemp-Hall.
Since the 2016 “experience” the school teaches pupils the test techniques for different types of SATs questions, so they “know the kind of explanation required”.
Julie McCulloch, primary specialist at the Association of School and College Leaders, said the 2017 reading test was already an improvement on the previous year. A group of psychometricians had helped design the test, and the STA had “been careful to choose texts that were more accessible.”
The STA had “worked constructively with teaching and leadership unions” about the standards, she said.
The changes were particularly welcome since the consequence of the 2016 tests had been “challenging for schools”, driving “quite punitive” accountability measures.
A spokesperson for the Department for Education spokesperson said this year’s key stage 2 results “showed the curriculum reforms are starting to raise standards.”
Reforms to the primary assessment system had “freed up teachers to educate and inspire young children, while holding schools to account in a proportionate and effective way.”
Ofqual also published a second report, which evaluated the STA’s approach to designing both the reading and maths tests.
The approach was robust, said the report – but some parts of the national curriculum, such as the need to “discuss and evaluate how authors use language”, might not be amenable to sample testing. This omission meant there was a “risk to validity” around these sorts of questions.
Ofqual used STA test data, its own investigations, independent experts and social media comments for the report.
Your thoughts