This is a book that wants to be taken seriously. It sets out to cut through the diverse opinions people have about education and provide an “evidenced based” overview of the way that children learn and the best ways to educate them. In the preface we are given an overview of what the authors consider to be appropriate standards of evidence, and each chapter ends with a detailed list of referenced sources. The scope is also huge, covering a brief history of education policy, from discussions on meta-cognition and gender differences, right through to academies policy and what PISA can tell us about education around the world.
Unfortunately, the authors are undone right from the start when they don’t share the method they used to select or reject the research included in their book. They are clear that they have been selective in their sources but do not explain their methodology or selection criteria; this consistently undermines the book’s claim to provide something more than just another person’s opinion.
For example, in the chapter on memory, the authors discuss a tool to boost working memory called Cogmed, suggesting that it may have long-term advantageous effects on academic performance, especially for children with ADHD. However, the only study cited to back up this claim is that carried out by the team that developed the Cogmed tool. A brief search reveals several papers and multi-study overviews, published in many of the same journals the authors cite elsewhere, that criticise this study and claim the Cogmed tool is actually ineffective.
The problem of inconsistent selection of sources occurs throughout. The authors sometimes reject studies for having poorly selected samples; at other times reference studies with tiny sample sizes conducted amongst undergraduates as evidence for how children learn. The result is that the reader is effectively asked to trust that the authors’ judgment is correct, rather than being provided with convincing evidence.
The authors also repeatedly break their own rules for what constitutes evidence. Despite declaring the fact that correlation does not equal causation is the “golden rule” of research, the chapter on comparative education is nothing but correlations in PISA and TIMSS data given causal meaning. For example, the fact that some countries near the top of the international rankings also have Confucian traditions is taken to mean that the Confucian values “appear to contribute” to the strong performance. This might turn out to be true, but the authors present no evidence beyond correlation for it.
Even more concerning is that all of the comparative analysis is done by just looking at the countries at the top of the rankings. As the Brookings Institute has pointed out, careful comparative research should explore across the distribution (bottom, middle and top) and see if particular structures or activities (such as Confucian traditions) are found more often at different points, not generalise from just the high performers.
The authors also refer to interventions that produced results that are statistically highly significant as producing “significantly higher” results. The two phrases mean very different things. High statistical significance only means the observed results are very likely to be replicated in the general population, while significantly higher suggests the size of the observed result is very large. Mixing these two up so completely is deeply misleading.
By far the most frustrating thing is that the authors are obviously capable of interesting analysis. The sections focused on psychology are the strongest, with the authors demonstrating impressive command of the subject and bringing in dissenting voices to be critiqued. There are also strong passages in the sections focused on education, there is a brief but good discussion on academies and free schools that I wished they’d spent more time on.
Ultimately, however, the flaws in the methods are too important to overlook and the scope proves too much. As an introduction to the psychology of learning this could be a useful book, but as an attempt to cover the whole of education and learning it loses its way, and as an evidence base to “support and possibly change teaching” it cannot be taken seriously.
Your thoughts