Questions over £1m ‘tutor-proof’ 11-plus tests

Questions over £1m ‘tutor-proof’ 11-plus tests

Durham University has earned at least £1 million for 11-plus tests brought in by local authorities to diversify grammar school intakes – despite accusations that there is no evidence to back up claims that they minimise the impact of tutoring and promote fairer access for pupils.

Documents released last week reveal the Centre for Evaluation and Monitoring (CEM) at Durham has generated £1 million from its new “tutor-proof” tests.

However analysis by campaign group Local Equal Excellent found the tests introduced in Buckinghamshire schools have made little difference to the number of state school primary pupils accessing secondary grammars.

Using data on success rates in the county – where all children sit the 11-plus – the group found the “tutor-proof test” made no difference to the large gap between pass rates of pupils from poor and wealthy areas, with the worst results among children on free school meals.

CEM has said it is too early to judge the test’s social impact, but the report has led to questions around the “tutor-proof” claim.

The university has rebuffed attempts by campaigners to extract potential evidence for the claims under the Freedom of Information (FOI) Act, including pilot test data and analysis. Its decision has been upheld by the independent Information Commissioner’s office (ICO), which adjudicates on such matters.

The adjudicator upheld the argument put forward by CEM that the disclosures would impact its commercial interests.

In a submission to the ICO, the university said its test had “grown rapidly in recent years and represented an income stream of around £1 million to the university”.

The ICO ruled that releasing the information would prejudice CEM’s unique selling point – its method of setting and scoring – which “made the test more resistant to coaching influences and question spotting”.

The report said the university had shown ways that tutors could manipulate the information if they were to learn facts about the test methodology held within the documents.

Peter Lockley, the ICO adjudicator in charge of the case, wrote: “There is public interest in protecting the methods by which CEM is attempting to deliver a fairer testing system, since divulging them would compromise a worthwhile experiment, before it had been given a fair chance to succeed.”

But Rebecca Hickman, an education consultant who helped to compile the Local Equal Excellence report, told Schools Week: “CEM should either publish their evidence or stop making claims that have no basis in fact.

“If they refuse to [publish their data], then we have every reason to be suspicious of their claims.

“CEM are, after all, part an educational institution who believe in certain standards of academic rigour – yet at every turn they are obstructing any kind of independent scrutiny of their so-called evidence.”

Dr Susan Stothard, head of assessment development at CEM, said the university did not claim that its tests were “tutor proof”, but developed to minimise the impact of “additional coaching”.

She said test questions required problem-solving and interpretation of novel information, rather than “simpler questions that can be answered through rote learning and quick recall”.

She told Schools Week: “Release of the test methodology could compromise the integrity of the tests, something that would not be in the best interests of the grammar schools or their candidates.”