News

Questions over £1m ‘tutor-proof’ 11-plus tests



Durham University has earned at least £1 million for 11-plus tests brought in by local authorities to diversify grammar school intakes – despite accusations that there is no evidence to back up claims that they minimise the impact of tutoring and promote fairer access for pupils.

Documents released last week reveal the Centre for Evaluation and Monitoring (CEM) at Durham has generated £1 million from its new “tutor-proof” tests.

However analysis by campaign group Local Equal Excellent found the tests introduced in Buckinghamshire schools have made little difference to the number of state school primary pupils accessing secondary grammars.

Using data on success rates in the county – where all children sit the 11-plus – the group found the “tutor-proof test” made no difference to the large gap between pass rates of pupils from poor and wealthy areas, with the worst results among children on free school meals.

CEM has said it is too early to judge the test’s social impact, but the report has led to questions around the “tutor-proof” claim.

The university has rebuffed attempts by campaigners to extract potential evidence for the claims under the Freedom of Information (FOI) Act, including pilot test data and analysis. Its decision has been upheld by the independent Information Commissioner’s office (ICO), which adjudicates on such matters.

The adjudicator upheld the argument put forward by CEM that the disclosures would impact its commercial interests.

In a submission to the ICO, the university said its test had “grown rapidly in recent years and represented an income stream of around £1 million to the university”.

The ICO ruled that releasing the information would prejudice CEM’s unique selling point – its method of setting and scoring – which “made the test more resistant to coaching influences and question spotting”.

The report said the university had shown ways that tutors could manipulate the information if they were to learn facts about the test methodology held within the documents.

Peter Lockley, the ICO adjudicator in charge of the case, wrote: “There is public interest in protecting the methods by which CEM is attempting to deliver a fairer testing system, since divulging them would compromise a worthwhile experiment, before it had been given a fair chance to succeed.”

But Rebecca Hickman, an education consultant who helped to compile the Local Equal Excellence report, told Schools Week: “CEM should either publish their evidence or stop making claims that have no basis in fact.

“If they refuse to [publish their data], then we have every reason to be suspicious of their claims.

“CEM are, after all, part an educational institution who believe in certain standards of academic rigour – yet at every turn they are obstructing any kind of independent scrutiny of their so-called evidence.”

Dr Susan Stothard, head of assessment development at CEM, said the university did not claim that its tests were “tutor proof”, but developed to minimise the impact of “additional coaching”.

She said test questions required problem-solving and interpretation of novel information, rather than “simpler questions that can be answered through rote learning and quick recall”.

She told Schools Week: “Release of the test methodology could compromise the integrity of the tests, something that would not be in the best interests of the grammar schools or their candidates.”



Your thoughts

Leave a Reply to Sally Jones Cancel reply

Your email address will not be published. Required fields are marked *

3 Comments

  1. Sally Jones

    Having just started looking at 11+ papers with my son, I think that the test, in particular the comprehension, rewards children who have been well taught and exposed to a rich variety of literature both at home and at school rather than crammed for the exam. I also think that it’s more common for state pupils to be tutored. So the results don’t surprise me. I suspect people would like a test that separates ability from attainment – but is that really possible?

    • Anita Crockett

      From what you said about the papers Sally this means that the new 11+ test still fails it’s objective. Buckinghamshire tutors are at full speed so little has changed. Non verbal reasoning tests (or full CAT tests), taken every 2 years is probably the best way of finding students with potential regardless of their schooling or background. They cannot be coached and track the child throughout school ensuring that a child does not just get lucky on one test.

  2. If the test identifies innate ability then why can pupil premium children score lower scores in Birmingham, Warwickshire and other areas?

    The system is a scam. Children remember content, pass it on to tutors who prepare children. No past papers means tutoring is how children gain past content.