Standardisation tests for SATs moderators will include questions written by artificial intelligence (AI) under a new government trial to cut costs and school workload.
The Standards and Testing Agency (STA) is conducting the pilot amid a push to “harness the power of AI technology across the sector”.
Key stage 2 moderators must pass a standardisation exercise to ensure they have the “required knowledge” to examine English writing teacher assessments.
Unlike maths and reading SATs, English writing is marked by teachers, with local authorities required to moderate 25 per cent of maintained school and academy papers.
Trial ‘to cut costs’
But moderators must pass one of three standardisation tests to get approval. This year, the third test will be produced using AI.
The pilot aims to “explore whether large language models (LLMs) can help overcome ongoing challenges with standardisation exercise production, including sourcing sufficient scripts from schools and reducing associated costs,” the government said.
Since 2021-22, the Australian Council for Educational Research has created the exercises, using real samples of year 6 children’s work.
But the Department for Education said “collecting suitable pupil writing samples is expensive and adds extra work for schools”.
The trial follows “18 months of research [into] how effectively LLMs can generate scripts that are representative of the work and ability of year 6 pupils”.
To produce the AI tests, STA’s “expert assessment researcher will design prompts” based on the teacher assessment framework to “generate draft pupil scripts”.
The agency will then “edit and carefully review” them to “ensure they meet our standards for accuracy, validity and reliability” before they are used in the test.
LAs can opt out
Local authorities can “opt out of participating … if they have concerns about the use of LLMs in the creation of the materials”. Their moderators will instead sit the other exercises.
Schools Week approached the 15 biggest councils in England to find out if they were taking part.
Four responded, with three saying they will be involved.
Kent, England’s largest local authority, which employs “around 60 moderators” who teach in primary schools or work as English subject leaders each year, said its reviews of the AI scripts found them “to be reliable”.
Dr Mick Walker, president of the Chartered Institute of Educational Assessors, said offering the AI tests “as an optional extension to normal procedures” meant it “does not pose a risk to pupils”.
“The inclusion of feedback presents an opportunity for moderators … to inform future developments.”
Any future decisions following the trial would be subject to the government’s response to the curriculum and assessment review panel’s final report.
A frequently asked questions document produced by the agency ahead of the trial said the DfE’s “overarching strategy is to harness the power of AI technology across the sector”.
However, the agency added it had “no plans to use AI to develop statutory national curriculum tests or assessments for primary pupils”.
AI’s ‘inevitable’ growth
Duncan Baldwin, an education consultant, said it was inevitable that AI would be used increasingly to support time-consuming and expensive tasks in education.
“AI can deliver huge benefits for teachers, school leaders and the government.”
Schools Week revealed last month that Ofsted inspectors trialled AI to save them time while taking notes during visits.
But the watchdog decided against any wider rollout as it felt the tools created more work, rather than streamline inspections.
The DfE also published AI toolkits for schools earlier this year. The guidance said leaders should plan for its “wider use” – including to analyse budgets and help plan CPD.
Ofsted also revealed that it used AI to process responses to its consultation on planned report cards “to improve efficiency”. It noted this was becoming “more common across government”.
Your thoughts