Exams regulator Ofqual has demanded evidence of improvements from the government’s SATs agency and contractor Capita after a series of failings during last years’ test series.
Schools Week investigations have revealed problems with the first SAT series run by outsourcing firm Capita under its £107 million, seven-year contract to oversee the management of the national tests.
Ofqual’s annual SATs report, published today, reveals the full extent of issues – including that 7,000 pupils did not get least one of their results on time.
Schools Week revealed 2,000 papers were still being missing at the start of this academic year.
Here’s what the regulator found…
‘Tech issues’ in marker training
Ofqual monitored the first stage of marker training, where a group of markers are invited to give feedback on the training materials.
In the first meeting, the regulator found there were “technological issues with online systems that meant some users were in the wrong virtual room”.
It also found “difficulties completing participant identification checks” because of “insufficient supplier resourcing”. Capita made changes after concerns were flagged.
Schools Week reported how markers spent hours trying to access a training webinar last year.
But in its report, Ofqual concluded there was “no evidence” that the quality of marker training was impacted by the move to online training.
1 in 5 primaries missing SATs results
While 99.5 per cent of results were sent out on results day, the STA had to send 3,360 primary schools – about 20 per cent – a letter warning marks in one or more subjects would be delayed for some of their pupils, the report says.
Ofqual added 7,437 pupils had at least one delayed result at this point – just over 1 per cent of all year six pupils.
About half of pupils received their results shortly after. But by August 26, the STA was unable to provide a result in one or more tests for about 1,700 pupils.
Just under 2,000 of the approximately 4,000 unaccounted for scripts were declared lost – compared to below 200 each year between 2015 and 2019.
Scanning technical issues to blame
Ofqual said delays were mostly caused by “technical issues arising from the scanning process”, meaning markers had to request re-scans of scripts.
Some test booklets also arrived late at the scanning site.
Ofqual said it was originally assumed by Capita that most missing scripts were because pupils were incorrectly registered for a test they did not take.
But the regulator said this was only true in a “small minority” of cases.
About 2,000 scripts were found during several weeks of physical searches of Capita’s scanning facility in late July and August.
Ofqual said scripts were lost “at least in part to failures in the reconciliation of scripts at the contractor’s scanning site”.
Nearly 400 wrong results issued
Schools Week reported how there were claims of incorrect marks being issued.
Ofqual’s report confirms there were 372 cases where pupils received a result for a test they did not take, or belonged to another pupil.
However, if this relates to the numbers of papers sat, it accounts for just 0.01 per cent of the 3.8 million taken last year.
The STA told Ofqual these cases were the “result of errors in the manual processing of pupil data”.
“The contractor was entirely reliant on schools to identify where these issues had occurred.”
‘Specific problems’ with tests for SEND pupils
Scripts can be modified for children with additional needs, such as a hearing or visual impairment. But Ofqual found “specific” problems with the scanning, marking and time return of some of these papers.
Just under 1,000 modified test scripts were affected. The reasons included scripts needing to be “reprocessed” before they could be marked, resulting in a delay.
In a “small number” of cases, schools returned a “blank ‘standard’ script” in addition to the completed modified script. The standard one was then processed before, or instead of, the modified one.
‘Excessive waits’ for helpline
Ofqual said schools faced “excessive waiting times” on Capita’s phone helpline.
Schools Week revealed how two-thirds of teachers hung-up during the SATs week in May.
Ofqual’s report confirms this, adding that the average response time was approximately 53 minutes.
“Many schools also reported that helpline staff were unable to resolve their queries,” the report reads, adding there was “insufficient stuff on the helpline and inadequate training”.
The email backlog generated during test week took four weeks to clear.
Ofqual requests evidence
Ofqual says it had warned the STA of the extra risks with a new contractor, online training and the pandemic hiatus.
The regulator has now “emphasised the need to identify the underlying causes” of the lost papers “to ensure that appropriate lessons are learned”.
Ofqual has set expectations of STA, and demanded evidence of the specific actions and process changes address the issues.
They also expect STA to consider where improvements to “risk management processes and management information might improve its ability to identify risks and issues at an earlier stage”.
Finally, STA has been asked to “consider how lessons from delivery in 2022 could inform its approach to working with any future delivery contractors”.
Capita told Schools Week that after feedback, training for markers has been enhanced, communications to schools have been improved and the test script processes have been strengthened.
Your thoughts