Last year’s key stage 2 SATs reading test was harder than those set in recent years, but Ofqual found “no evidence” that it “failed to meet its stated purpose”.
The exams regulator has published analysis of the test, which last year prompted complaints from parents and teachers that it left pupils in tears.
Nick Gibb, the then schools minister, insisted at the time that he was “confident the test was set to an appropriate level of difficulty”.
The results of the analysis have been published as part of Ofqual’s national assessments regulation annual report for 2023.
Here’s what we learned.
1. Reading test was harder than recent years…
Today’s Ofqual report found the 2023 reading test was harder than those set in recent years, but not as difficult as the 2016 reading test, which was found to be “unduly hard”.
The score threshold for this year’s test was 24, “indicating that its difficulty falls between the 2016 test and the tests administered since”. The threshold in 2016 was 21, and it has ranged between 26 and 29 since then.
The evidence also “suggests that lower attaining pupils, in particular, were likely to have experienced the 2023 test as more difficult than tests in the years after 2016”.
2. …but ‘met its purpose’
Ofqual said it was “important to note that perceptions of test difficulty on the one hand, and validity and reliability on the other, are two different issues”.
“If a test is, or is perceived to be, more difficult or easy than the same test in previous years, this does not necessarily mean that the test results are more or less valid or reliable than those in previous years.”
The analysis “found no evidence that the test failed to meet its stated purpose of ascertaining what pupils have achieved in relation to the attainment targets outlined in the 2014 national curriculum”.
The report added that “outcome data provided by STA suggests that the test was effective in differentiating across the ability range”.
3. Three in 10 didn’t finish
Three in ten didn’t complete the last question on the test, higher than at any point since 2018 (the figures for 2016 and 2017 are not comparable because in those years, the last item was a selected response question).
But Ofqual said that, for assessments of this nature, “it is neither surprising, nor necessarily problematic for reliable and valid results, that not all pupils will reach the end of the test”.
“Nevertheless, there are legitimate questions to ask about the point at which the omission rate for the last items is considered too high. STA should consider this question carefully in consultation with relevant experts in future test development.”
4. Delay to marking didn’t affect standards
There were a number of technical issues with the marking and results process last year, through not to the extent seen in 2022, the first year the tests were run by Capita.
The start of marking was delayed by a week “owing to an error in the contractors’ systems such that the first round of preparatory work for marking had to be restarted”.
But Ofqual concluded this “did not prevent STA from completing standards maintenance procedures normally and on schedule”.
5. Issue with gateway ‘relatively short-lived’
When results were returned to schools via the online primary assessment gateway on July 11, “some schools and teachers reported that they were unable to access their results at 7.30am when the PAG opened”.
“By 10.30am 13,238 schools (approximately 79% of primary schools in England) had downloaded their results.”
Ofqual said the issue was “relatively short-lived and was associated with high system demand, with few reports of issues from teachers and schools after midday”.
6. Overall delivery ‘successful’
Ofqual concluded that “overall in 2023, national assessment delivery was successful”.
“Significant progress” was made in relation to the issues documented in Ofqual’s national assessments regulation annual report for 2022.
On results day, the STA returned more than 99.9 per cent of results to schools, compared to over 99.5 per cent the previous years.
There were 353 pupils with missing scripts, although 60 pupils “still achieved the standard based on their performance in other papers”.
“Between 2015 and 2019 script losses had been below 200 but in 2022 they reached nearly 2,000.”
The average wait time for the helpline also fell from 53 to 22 minutes.
7. Most markers stressed and underpaid
Ofqual also surveyed markers for the first time.
Although 80 per cent said they had a good working relationship with supervisors and 70 per cent found their role challenging, 60 per cent of markers reported they found their role “stressful”.
Almost 40 per cent of markers said their workload was not realistic, while more than 65 per cent said their pay was not satisfactory.
It comes after Schools Week revealed how Capita – paid £107 million to run SATs – had again slashed rates for markers, who said their pay packets remained the same despite a doubled workload.
However, Ofqual found fewer than one in five respondents indicated they were unlikely to continue marking.
Your thoughts