Accountability measures for secondary schools are ‘flawed and dangerous’, new analysis suggests

Secondary school’s published performance measures will this year be “very flawed and quite dangerous” due to a series of anomalies, a union policy director has claimed.

But the government and schools inspectorate Ofsted have both said the issues will be taken into account and flagged to the public as necessary.

Several education professionals contacted Schools Week after provisional data was sent to secondary schools last week via RAISEonline.

The data allows schools to compare their GCSE performance with other schools, including data on pupil progress.

The decision by two teacher unions in 2010 to shun national primary tests at key stage 2 appears to be negatively impacting secondary schools that taught large numbers of pupils caught in the boycott. About 25 per cent of pupils were affected.

Boycott pupils received a “teacher-assessed grade” for English and maths combined between 2.5 and 5.5, and increasing in increments of 0.5. Examined pupils received “fine grades” across the same range but increasing in incerements of 0.1 (see graphic).

Results-graph-1

Schools Week has seen analysis by Professor David Jesson, associate director of University of York’s Centre for Performance Evaluation, showing a much higher number of teacher-assessed pupils were given the highest mark.

Results-graph-2

Almost a quarter of pupils in boycott schools received the top grade (22 per cent) compared with around 10 per cent in schools taking the tests.

Pupils assessed at level 5.5 or above are expected to achieve top-end GCSE results, such as A/A*.

In some areas high numbers of pupils were in the boycott. In Hartlepool, for example few, if any, year 6 pupil took the tests.

Professor Jesson said this was likely to affect the progress measure (also known as “value-added scores”) of their secondary schools.

He said: “Schools need to be aware they will have been misrepresented in this data.

“Where there are schools with lots of pupils who have not had proper assessment, their value-added is unlikely to reflect the actual picture.”

He said this could affect Ofsted’s perceptions of schools and might mean schools were unfairly caught under proposed interventions for “coasting” schools.

The Association of School and College Leaders’ deputy director of policy, Duncan Baldwin, said: “This year’s data is potentially very flawed and quite dangerous. Extreme caution should be given to using this for any accountability measures such as Attainment 8, Progress 8 and the coasting measures.”

Seventy-nine per cent of GCSE pupils at Humphry Davy school in Penzance came from schools that boycotted the 2010 SATs.

Headteacher Bill Marshall said he wanted to make other schools aware of the issue.

“Five years ago a decision was made by staff in primary schools because of challenges put forward to them by the government and concerns over the tests. Over time, it did result in changes to assessment procedures and there was good reason.

“But, consequently, to judge secondary school performance on this data could leave them vulnerable to a ‘requires improvement’ judgment from Ofsted if inspectors do not take into account the underlying causes for dips in value-added performance.”

After Schools Week raised the matter with the Department for Education, a spokesperson revealed that it had “taken steps to ensure” the public was aware of potential discrepancies.

In the 2015 performance tables they “will be making it clear where more than 50 per cent of the key stage 2 results used to calculate the value-added were based on teacher assessments”.

Both unions responsible for the 2010 boycott deny it has caused issues for schools.

The general secretary of the National Union of Teachers, Christine Blower, said that schools will be measured under new progress measures “on the basis of data for three consecutive years, not one”.

The general secretary of the National Association of Head Teachers, Russell Hobby, said: “There are so many simultaneous changes to exams, grade boundaries and measures that it would be difficult to ascribe any effect to the 2010 boycott. Ofsted would do well to treat all data in 2015 with a high degree of scepticism.”

An Ofsted spokesperson said “Our inspectors are used to dealing with statistical anomalies and cases where data might not be reliable, so we have every confidence that schools will not be disadvantaged over this issue. We have been discussing the issue with stakeholders and will shortly be issuing guidance to inspectors on this specific matter.”

English language grades are also firing concerns. In August, the national average pass rate was announced by the Joint Council for Qualifications as being 67 per cent, but the recently released data describes it as 83 per cent.

Schools previously thought to be above the national average now look “significantly below” average, one teacher told us. It is understood the higher national average excludes iGCSE results and combined English GCSEs.