Billy Beane, manager of small-budget underdog baseball team the Oakland As, was frustrated at once again losing to the (much richer) New York Yankees in the playoffs.
Not only did the Yankees go on to win the World Series, but they also then bought the As’ best players. “There are rich teams, there are poor teams, then there’s 50 feet of crap, and then there’s us,” laments Beane (Brad Pitt) in the movie adaptation of Michael Lewis’ book, Moneyball.
Every school leader knows how Beane felt, because each has experienced the struggle of attracting or holding on to those with skills that are in demand in the private sector. Pay rises will help, but won’t solve the fundamental problem.
Beane never solved his either. The As never won the World Series. But we can perhaps learn something from what he did next.
Until then, baseball player recruitment had been driven by scouts – wise old owls who knew a good ball player when they saw one. Beane tore up that model and embraced analytics.
The As hired economists who snapped up players their analysis said were undervalued by the market. It didn’t win them the World Series, but they did break the record for the most consecutive wins. More importantly, the As’ success transformed the way baseball recruited players forever.
Sadly, we couldn’t copy the As even if we wanted to. For a start, we don’t have lots of unwanted teachers desperate to step up to the plate.
More importantly, unlike pre-analytics baseball, we have no reliable quantitative measure of teacher impact. This means that like pre-analytics baseball, we’re left with received wisdom and various proxies for effective development and practice.
There probably are some wise senior leaders who know impactful teaching when they see it, but we need to do better.
Can we make tacit knowledge explicit? Can we use that knowledge to transform teacher development, in the same way that analytics transformed baseball?
We couldn’t copy the As even if we wanted to
The National Institute of Teaching (NIoT) believes we can. US researchers have estimated teacher impact by measuring progress in standardised test scores from one year to the next (known as “value-add”), but this is rarely possible in England because other than SATs and GCSEs, schools use different end-of-year assessments.
However, schools in large trusts often follow the same curriculum and use the same assessments, making analysis of teacher impact possible.
We worked with our founding trusts to build the Teacher Education Dataset (TED), comprising anonymised assessment results of 115,000 pupils from 180 schools, linked to their 7,000 teachers, without identifying them, making value-add analyses possible.
Unlike in baseball, this is not – and never will be – a performance management tool. All the data is anonymised. However, there’s enough of it that we may be able to see patterns in pupil attainment that might tell us something about teacher impact in those trusts.
To that end, Professor Rob Coe analysed these records to see if some classes regularly outperformed statistical predictions based on their pupils’ prior attainment and characteristics (such as eligibility for Free School Meals or SEND status).
Although we cannot, and should not, identify the teacher, this information allows us to learn that there is indeed a small number of teachers who we can say with statistical confidence have a higher positive impact than others.
Like any research, there are nuances to this finding and we need to take care in how we describe it. Nevertheless, we believe our measure can help increase teacher impact throughout the system.
So, over the next four years and with support from the Nuffield Foundation, we will be working to learn more about the characteristics of high-impact teachers in their trusts , their classroom practices and the schools they work in.
We will explore how teacher impact increases over time and whether some training pathways generate more impactful teachers than others. We also hope to use AI analyses of observation records to learn whether some practices are more impactful than others, and we will share our findings with everyone.
Just like the As, there may not be a fairytale ending. Analytics will not be a quick fix, and teachers and teacher educators will need to continue their hard, painstaking work in developing themselves and others.
But better understanding teacher impact and development could dramatically increase our understanding of what makes a good teacher, and keeps them in the profession.
And that would be a home run.
Your thoughts