The exams for some technical qualifications are too easy, too hard or not reliable enough, Ofqual has warned.
The watchdog has looked at how trustworthy the exams for technical and vocational qualifications are, and found that 29 per cent of them do not stand up to scrutiny.
In recent years, questions over the rigour of vocational qualifications, and the creation of more rigorous criteria for their inclusion in performance tables, have prompted some exam boards to introduce externally-assessed examinations.
Today’s report reveals how Ofqual looked at a sample of externally-marked exams across 27 vocational qualifications, all of which counted or will count in 16-to-19 performance tables.
Most of the qualifications were at level one or two, but some level three qualifications were also included, in subjects such as health and social care, carpentry, hospitality, digital media, and applied science.
Out of the 49 exams Ofqual scrutinised, 14 are not “functioning” as a trustworthy reflection of pupil performance, it said.
The 14 exams were either too easy or too hard, though some also had too many or too few questions for the time limit.
For example, the average proportion of total marks achieved by pupils varied from 23 per cent in one exam, meaning it was too difficult, up to 80 per cent in others, meaning it was too easy.
Easy tests with too many pupils scoring highly “would be unlikely to provide adequate differentiation between students of different abilities.”
The exams are also less reliable than GCSEs on average, Ofqual’s research revealed. Where GCSE exams have a reliability measure of 0.8, on average these exams had a reliability measure that was slightly lower, at 0.74.
A reliability measure of 0.5 is generally considered too low, yet eight per cent of the vocational and technical exams scored below this threshold.
In fact, three exams had lower levels than any seen in a previous study of reliability for GCSEs and A-levels.
Overall, seven awarding organisations had samples of their exams looked at by Ofqual. All of them have received reports on how their tests fared, but Ofqual refused to share their identities with Schools Week.
The report was released to coincide with an Ofqual event in London, which focuses on vocational and technical qualifications and aims to “raise awareness and understanding of the qualifications landscape”.
Attendees will discuss the challenges and opportunities of reforming their qualifications, the regulator said.
Your thoughts