The government will correct the record over its claims about the effectiveness of academy trusts after being slapped down by the statistics watchdog.
The Office for Statistics Regulation, the regulatory arm of the UK Statistics Authority, has raised issues about the “transparency, quality and replicability” of data in a document published alongside the schools white paper in March.
The watchdog launched an investigation in April following a complaint from the National Education Union, which accused the government of “misreporting” data to back its academies reforms. The department cherry-picked trusts for comparison in its “case for a fully trust-led system” report, NEU said.
In letter to Department for Education chief statistician Neil McIvor, OSR director-general Ed Humpherson welcomed the department’s “plans to update the document”.
He said it was “not always possible” to identify the exact data used to produce analysis in the document. Links to sources of data “should be clearly set out and enable users to easily find the specific data referenced”, he added.
‘Insufficient info’ on methodologies used
He added that there was “insufficient information” on the methodologies used to produce statistics, and warned the “limitations of these methodologies and the implications that these would have on the fairness of the comparisons being drawn have not been fully explained to users”.
The DfE should include “clear information on the methodologies and associated caveats so that users can draw reliable conclusions”.
Finally, he said that “limited transparency” around the data sources and methods meant it was “difficult for users to replicate the figures presented and to draw their own conclusions”.
The department should “ensure that sufficient information is included in the document so that users are able to easily replicate the statistics”.
Humpherson acknowledged the document was “not a purely statistical document”, but warned the approach “risks conflating the policy proposals with the statistical analysis presented”.
DfE ‘planning on making changes’
“Care should be taken when producing these types of documents to ensure that the statistics are presented clearly and independently of the conclusions drawn from them.”
He said he understood the DfE’s statisticians were “planning on making changes to the evidence document as well as publishing an additional document which will contain the data and associated methodological information”.
This additional information “will be valuable to users and will help inform the public debate and we are pleased to see the swift action taken by your team to make this further information available”.
Dr Mary Bousted, joint general secretary of the NEU, said the government “claims that its plan to force all schools to join a multi-academy trust is evidence led”.
“However, after months of preparation the document it came up with was so flawed that the Office for Statistics Regulation have agreed with the NEU’s complaint that the report is misleading.”
‘No evidence’ MAT drive will improve schools
She claimed the decision showed there was “no evidence that forcing all schools to join a multi-academy trust will improve schools”.
“It is time for Nadhim Zahawi to withdraw his plan, which is now exposed as wholly ideological, and get back to what teachers and parents actually want the government to focus on.”
A DfE spokesperson claimed its evidence “remains clear that strong multi-academy trusts have a good track record of improving underperforming schools as sponsored academies”.
“We welcome this input from the Office for Statistical Regulation and have updated our ‘Case for a fully trust led system’ document to ensure even greater clarity and transparency, noting the department’s conclusions relating to academies’ performance remain the same.”
However, the document does not yet appear to have been updated.
The DfE has been slapped down over its use of data several times in recent years. Between 2017 and 2019, the department was reprimanded five times by the watchdog, with several warnings relating to its claims about school funding.
That sadly is what our government does. They write lies on buses, tell lies about our NHS, Energy plans, our education system, number ofhome educators and if they follow the rules during lockdown to name but a few. Never have I been so aware of how untrustworthy our government is. It’s shocking!
Academies do not improve education for all children. They may lead to higher levels of unofficial selection or exclusion. They may lead to ‘off rolling’ in order to pretend to improve exam results. They do lead to fragmentation, rather than local schools working together. I have not been keeping my ‘academy problems’ record up to date but I was tracking some of the issues which were reported: https://www.dropbox.com/s/1mxepto2zjq5x6c/Academy%20Problems.xlsx?dl=0