Long-awaited EEF phonics study shows ‘disappointing’ progress

Children who learn to read using the country’s “leading” phonics programme make one month’s additional progress compared to their peers, while older pupils using a linked catch-up programme typically fall at least two months’ behind.

That is the “disappointing” conclusion of a long-awaited independent evaluation by the Education Endowment Foundation (EEF) of the Read Write Inc and Fresh Start schemes, offered by Ruth Miskin Training and used by 8,000 schools.

However, the £1 million evaluation was beset with problems, including a second trial being canned because of the pandemic, with the EEF urging caution on interpreting the findings.

What did the phonics study find?

The EEF found children aged four to nine who participated in the Read Write Inc. Phonics (RWI) programme on a daily basis made, on average, an extra month’s progress in reading compared to children in the control group.

Meanwhile, older children aged nine to 13 who took part in Fresh Start (FS), a daily catch-up phonics intervention for those below their expected reading age made two months’ less progress.

In both situations, the impact of the programme was exaggerated for children eligible for free school meals.

Disadvantaged pupils made, on average, three months more progress than their peers when they participated in RWI. Meanwhile older disadvantaged pupils typically fell three months behind their peers when they participated in FS.

More than 130 primary schools were recruited by Queens University Belfast to take part in the trial, which began in 2016.

The control group consisted of 65 schools, which continued with their ‘business as usual’ reading provision.

‘It’s a failed trial’

However, EEF itself said the findings – particularly those just looking at disadvantaged pupils – should be “interpreted with caution”.

Many pupils were not included in the final analysis due to factors including absence. In the FS trial, more than a third of schools offered the intervention did not even deliver the programme at all.

A spokesperson for Ruth Miskin said the trial did not meet EEF’s “high-quality standards… and are not a true reflection” of the impact of its schemes.

The draft EEF report rated the RWI finding as having “low to moderate” security, with a “moderate” security for its FS conclusion.

Professor Stephen Gorard, director of the Durham University Evidence Centre for Education, rated the “trustworthiness” as one out of five.

“I would say it’s a failed trial. I think the amount of missing data means we can’t really draw any conclusions.”

Schools want ‘trustworthy’ evidence

Schools are looking for interventions “which have trustworthy evidence and a big bang. This has neither,” he said. “It doesn’t mean the underlying interventions aren’t any good. It’s just that we can’t and we shouldn’t do anything on the basis of this trial.”

The average costs of RWI for one school was around £18,960, or £186 per pupil per year, when averaged over three years, the report found.

However, the report also looked at relative cost-effectiveness. The EEF found that, on average, the relative cost-effectiveness of RWI for one month’s progress is £126,400 per school per year, or £3,718 per pupil per year.  

“If this figure is accurate, then this programme is very, very expensive, with consequences for the use of public funding of schools,” said professor Dominic Wyse, who recently co-authored a landmark study on the teaching of phonics and reading.

Fellow academic Alice Bradbury, professor of sociology and education at University College London, added an extra month’s progress for pupils who participate in RWI was “disappointing” given its popularity.

But the Ruth Miskin spokesperson said “schools that teach our programme with fidelity achieve impressive results”, pointing out 20 of the 34 English hubs use the programme and that “Ofsted reading deep dives praise the quality of teaching in schools that teach our programme”.

Second Read Write Inc trial canned

But the findings are likely to call into question the EEF’s decision to delay publishing results until a second report into the scheme was completed.

The latter study started in 2019 to evaluate RWI’s delivery through the Teaching and Leadership Innovation Fund.

But it was “significantly disrupted” by the pandemic. It was announced today that no findings will now be published. Instead, a “lessons learned” report has been produced.

Professor Becky Francis, EEF chief executive, said “robust evidence suggests that high-quality, structured phonics teaching can boost young pupils’ literacy development, when they are implemented carefully and as part of a wider literacy offering.   

“Going forward, we need more research around the impact that phonics can have on older pupils. Building the evidence base further will help us to better understand the impact that phonics approaches have on this age group.”