The self-professed “evidence-led education secretary” used data to back his all-academy push that his own statisticians admitted this week “may be misleading”.
Labour has said that Nadhim Zahawi must now return to Parliament to “correct the record” over his claims that high-performing trusts “deliver the best outcomes”.
The Department for Education was forced to amend and provide additional evidence for a document that makes the “case for a fully trust-led system” after criticism from the Office for Statistics Regulation (OSR) this week.
The watchdog raised issues about the “transparency, quality and replicability” of data in the document, which was published alongside the schools white paper in March.
But a new spreadsheet published by the DfE, supposedly to back up its claims, has cast further doubt on statements made by ministers.
A central government claim is that pupils in the top multi-academy trusts perform better than those in local authority-maintained schools.
But, in sections of the spreadsheet that show the relative performance of pupils in different schools, the DfE itself warns that “direct comparisons between LAs, MATs and SATs may be misleading because the characteristics of the schools are different”.
This has called into question claims made by Nadhim Zahawi in a Parliamentary debate on March 28, when he said the annex to the white paper showed “evidence that strong, high-performing multi-academy trusts really do deliver the best outcomes”.
‘Evidence-led’ Zahawi’s claims ‘flawed’
Shadow schools minister Stephen Morgan said the education secretary “likes to claim he is evidence-led, but his dodgy data is clearly flawed”.
He added: “He should return to the House, explain himself and correct the record. The government are obsessing over structures without evidence to show this will deliver better outcomes for children in the classroom.”
Labour plans to raise a point of order with the speaker Sir Lindsay Hoyle next week, Schools Week understands.
It is not the first time the DfE has admitted that comparisons between different types of school are not always clear-cut.
In its 2018-19 academies annual report, the department warned that “making fair comparisons between sponsored academies and LA schools is complex”.
This is because the sponsored academies programme targeted poorly-performing schools, which in turn raised the “average quality of the remaining LA-maintained schools”.
But conversely, many high-performing LA-maintained schools have become converter academies and “this can act to reduce the average quality of the remaining LA maintained schools”.
This week’s slapdown from the statistics watchdog follows a complaint from the National Education Union, which accused the government of “misreporting” data to back its academies reforms. The department cherry-picked trusts for comparison in its report, the NEU said.
Stats watchdog criticises academy evidence
In a letter to DfE chief statistician Neil McIvor, OSR director-general Ed Humpherson said it was “not always possible” to identify the exact data used to produce analysis in the document.
Links to sources of data “should be clearly set out and enable users to easily find the specific data referenced”, he added.
He added that there was “insufficient information” on the methodologies used to produce statistics.
The “limitations of these methodologies and the implications that these would have on the fairness of the comparisons being drawn have not been fully explained to users”, the letter added.
The DfE should include “clear information on the methodologies and associated caveats so that users can draw reliable conclusions”.
Finally, he said that “limited transparency” around the data sources and methods meant it was “difficult for users to replicate the figures presented and to draw their own conclusions”.
Dr Mary Bousted, joint general secretary of the NEU, claimed the decision showed there was “no evidence that forcing all schools to join a multi-academy trust will improve schools”.
But a DfE spokesperson claimed its evidence “remains clear that strong multi-academy trusts have a good track record of improving underperforming schools as sponsored academies”.
“We welcome this input from the Office for Statistical Regulation and have updated our ‘Case for a fully trust-led system’ document to ensure even greater clarity and transparency, noting the department’s conclusions relating to academies’ performance remain the same.”
The DfE has been criticised over its use of data several times in recent years. Between 2017 and 2019, it was reprimanded five times by the watchdog, with warnings relating to its claims about school funding.
Your thoughts