The Department for Education will change the methodology for its Progress 8 accountability measure in 2018, after a consultation revealed school leaders are angry over the “disproportionate” impact of outliers on their overall scores.
The changes will be made in response to “feedback from schools about the impact that a small number of pupils with extremely negative progress scores can have on a school’s overall Progress 8 score”, according to the DfE’s ‘School and college performance tables statement of intent’ for July 2017.
The DfE said it would work with the sector to find the best approach this autumn.
It has also promised to inform those working with school data, including Ofsted, regional schools commissioners and local authorities, of the potential impact on a school’s Progress 8 score of pupils with “extremely negative progress scores”.
The impact of these pupils’ scores on Progress 8 will also be “taken into account where a school is below the floor standard or coasting in 2017”.
The Progress 8 initiative was introduced in 2016 to try and capture all pupils’ average progress from the end of primary school to the end of secondary school, information that is then translated into a rating for the school, but concerns have began to surface.
Former head Tom Sherrington, for example, described the measure as an “embarrassment” that the government should “ditch“, in an article for Schools Week earlier this month.
The DfE also published research yesterday on schools’ views of the measure’s implementation, which found interviewees torn between the interests of individual children and their school’s overall result.
The ‘understanding schools’ responses to the Progress 8 accountability measure report found that every one of the 21 schools and 38 leaders interviewed are concerned by the “impact of outliers”, on their Progress 8 scores.
By this they mean the pupils who do not achieve as expected at GCSE due to “illness, bereavement, or referral to alternative provision” who have a “disproportionately negative” effect on a school’s overall score.
These effects are especially felt in smaller schools or those that already have negative Progress 8 scores, participants said.
School leaders also identified a “lack of contextualisation”, claiming that current Progress 8 calculations do not allow for variation due to special education needs, English as an additional language, or areas of deprivation.
The report found interviewees from schools with negative Progress 8 scores felt that pupils’ best – rather than their first – assessment entry should be counted, to reflect those who achieve higher grades in resits, for example.
Overall, 34 of the 38 interviewees claimed their understanding of the Progress 8 measure was ‘good’ or ‘excellent’, but further discussions revealed “inaccuracies or misconceptions” when discussing technical details.
The report concluded that “further guidance and support is required”, and recommended providing more support for schools on the calculation, as well as information on which combinations of qualifications have been approved.
CooperGibson Research carried out the study, conducting telephone interviews with 38 individuals across 21 schools. Twenty-five participants were senior school leaders (headteachers, deputy headteachers/vice principals), and 13 were middle leaders (heads of department, data managers).
They represented a range of academies and local authority maintained schools, with 17 mixed, three girls’ schools, and one boys’ school across the country.