The Pupil Premium Awards benefit schools that had a little more talent and a lot of luck
Nobel Prize winner Daniel Kahneman, when asked to report his ‘favourite equation’, offered the following: Success = talent + luck. Great success = a little more talent + a lot of luck.
He illustrates this with an example from sport. In a multi-day professional golf tournament, he notes, some players will always score exceptionally well on the first day.
Most casual observers will make assumptions as to why this has happened. The golfers must be more talented than others. The golfers are better, or have better coaches.
When discussing their performance, commentators say golfers have practised a lot, or have coaches who work wonders, or have sorted out some minor aspect of their games.
But in fact, none of these explanations are likely to be true.
These kinds of explanations are things people commonly think contribute to performance but are actually attributions based on performance. This is well known in some circles, so much so that it has become known as the halo effect, since all kinds of erroneous assumptions are made about successful performers.
The explanation for the golfers’ success is much more mundane than expected. The golfers were lucky. And in subsequent rounds, their scores will be, in almost
every case, closer to the average. This effect – the regression to the mean – was first noticed over a hundred years ago. Numbers which are out of the ordinary tend to regress towards an average expectation.
It simply isn’t possible to get lucky every year
Similarly, the Pupil Premium Awards reward luck. Since the awards began, schools in England have been automatically entered. The rules of eligibility, as clearly explained on the website, are simple: ‘All state-funded primary, middle and secondary schools in receipt of pupil premium funding with published key stage 2 or key stage 4 data will be automatically entered where you have an overall Ofsted inspection judgement for effectiveness of 1 or 2.‘
Until recently, schools could win cash prizes “of up to £10,000”. These were given to the schools that had seemingly most boosted the attainment of their most disadvantaged pupils. The problem with this kind of award is that those who have the best numbers are assumed to have done something unusual which has contributed to those numbers – even if they have helped children in similar ways to hundreds of other similar schools.
But the regression to the mean shows that, actually, schools which have unusual numbers simply had a year when they had ‘a little more talent + a lot of luck,’ as Kahneman suggests. Even now that schools have to show unusual results over three years rather than one to win up to £250,000, some schools somewhere will have got lucky three years in a row.
Sometimes golfers get lucky more than once too. In last year’s British Open, Rory McIlroy finished in first, second and third in the first three rounds. He could only manage to finish joint 35th (with 13 other players) out of 72 in the final round. It was a thoroughly average performance as he regressed to the mean. As a modern professional, McIlroy is also more aware than most about the reason for his win, saying afterwards: “I just needed something to click. Luckily everything clicked.” In the Pupil Premium Awards, luck seems to have been ignored.
This is a huge pity, because schools across the country are working hard, all the time, to support all their children, whether they win an award or not. The Pupil Premium was introduced to help disadvantaged children and the schools in which they are educated. It means schools are being left out because those in charge of policy simply don’t understand the halo effect, or relatively simple concepts such as regression to the mean.
I wish the winners well, but I would be surprised to see a school win more than once. It simply isn’t possible to get lucky every year. Even professional golfers struggle to win the Open repeatedly,
and only 150 or so of them compete each year. There are 24,000 schools in the country working hard for their children. It’s somewhat depressing to find their work being turned into some kind of luck-based game show, and a sad reflection on current education policy and thinking.
Jack Marwood is an education writer. He writes about data and school-related matters on his blog Icing on the Cake
Completely agree…also, possibly the most meaningless accountability measure ever invented.
Of course Alfie Kohn and Kahneman would also point out that such ideas are delusional beyond words. As Kahneman’s old friend Tversky said, “It’s frightening to think that you might not know something, but more frightening to think that, by and large, the world is run by people who have faith that they know exactly what is going on.” It’s something I try and tackle in my new book, From School Delusion to Design (out in spring)
So why does Rory McIlroy win more golf tournaments than others? Its not regression to the mean if its away from the mean, most peoples average number of major golf is tournaments is zero?
Your question is a red herring, as a reading of my piece will show. This isn’t about winning every time, or winning more often than others. Regression to the mean is the phenomenon that if a variable is extreme on its first measurement, it will tend to be closer to the mean on its second measurement.
Rory McIlroy’s average score is 70.308 shots per round (http://www.pgatour.com/players/player.28237.rory-mcilroy.html/statistics as of 18 December 2015). He had a standard deviation of 3.33 shots (http://fivethirtyeight.com/datalab/tiger-woods-was-right-about-rory-mcilroys-inconsistency-sort-of/ July 29 2014). This means he only scores less than 67 in around 1 in 6 rounds he plays.
It was therefore unusual for McIlroy to score consecutive rounds of 66, 66 and 67, as he did in the first three rounds of the 2014 Open. 67 was a regression to his mean of 70.308. His final round was 71, which was even closer to his mean score.
So he regressed to the mean over the course of the tournament, as I said above.