The Pupil Premium Awards benefit schools that had a little more talent and a lot of luck

Nobel Prize winner Daniel Kahneman, when asked to report his ‘favourite equation’, offered the following: Success = talent + luck. Great success = a little more talent + a lot of luck.

He illustrates this with an example from sport. In a multi-day professional golf tournament, he notes, some players will always score exceptionally well on the first day.

Most casual observers will make assumptions as to why this has happened. The golfers must be more talented than others. The golfers are better, or have better coaches.

When discussing their performance, commentators say golfers have practised a lot, or have coaches who work wonders, or have sorted out some minor aspect of their games.

But in fact, none of these explanations are likely to be true.

These kinds of explanations are things people commonly think contribute to performance but are actually attributions based on performance. This is well known in some circles, so much so that it has become known as the halo effect, since all kinds of erroneous assumptions are made about successful performers.

The explanation for the golfers’ success is much more mundane than expected. The golfers were lucky. And in subsequent rounds, their scores will be, in almost
every case, closer to the average. This effect – the regression to the mean – was first noticed over a hundred years ago. Numbers which are out of the ordinary tend to regress towards an average expectation.

It simply isn’t possible to get lucky every year

Similarly, the Pupil Premium Awards reward luck. Since the awards began, schools in England have been automatically entered. The rules of eligibility, as clearly explained on the website, are simple: ‘All state-funded primary, middle and secondary schools in receipt of pupil premium funding with published key stage 2 or key stage 4 data will be automatically entered where you have an overall Ofsted inspection judgement for effectiveness of 1 or 2.‘

Until recently, schools could win cash prizes “of up to £10,000”. These were given to the schools that had seemingly most boosted the attainment of their most disadvantaged pupils. The problem with this kind of award is that those who have the best numbers are assumed to have done something unusual which has contributed to those numbers – even if they have helped children in similar ways to hundreds of other similar schools.

But the regression to the mean shows that, actually, schools which have unusual numbers simply had a year when they had ‘a little more talent + a lot of luck,’ as Kahneman suggests. Even now that schools have to show unusual results over three years rather than one to win up to £250,000, some schools somewhere will have got lucky three years in a row.

Sometimes golfers get lucky more than once too. In last year’s British Open, Rory McIlroy finished in first, second and third in the first three rounds. He could only manage to finish joint 35th (with 13 other players) out of 72 in the final round. It was a thoroughly average performance as he regressed to the mean. As a modern professional, McIlroy is also more aware than most about the reason for his win, saying afterwards: “I just needed something to click. Luckily everything clicked.” In the Pupil Premium Awards, luck seems to have been ignored.

This is a huge pity, because schools across the country are working hard, all the time, to support all their children, whether they win an award or not. The Pupil Premium was introduced to help disadvantaged children and the schools in which they are educated. It means schools are being left out because those in charge of policy simply don’t understand the halo effect, or relatively simple concepts such as regression to the mean.

I wish the winners well, but I would be surprised to see a school win more than once. It simply isn’t possible to get lucky every year. Even professional golfers struggle to win the Open repeatedly,
and only 150 or so of them compete each year. There are 24,000 schools in the country working hard for their children. It’s somewhat depressing to find their work being turned into some kind of luck-based game show, and a sad reflection on current education policy and thinking.

 

Jack Marwood is an education writer. He writes about data and school-related matters on his blog Icing on the Cake