A new paper looks at what we know about teachers’ evaluation skills and their impact on outcomes. Jonathan Haslam unpicks the detail
The answer to this article’s titular question might seem obvious, but a new systematic review has revealed just how little we know about teachers as evaluators.
According to the review, evaluation refers to the systematic investigation of the effectiveness, worth, or merit of a program, practice or policy. Ultimately this leads to a judgement about the value of a particular approach. The value of an approach might not necessarily be in terms of academic outcomes for students. It might be its impact on social-emotional outcomes, staff workload, financial cost, or whatever value is important to the teacher or school. Does the approach “work” to improve that outcome?
Teachers and schools are informally evaluating particular approaches all the time. Even in these crazy days of home learning and nearly empty schools, teachers are continually weighing what’s worked well and what hasn’t, and how things could be improved tomorrow. The question is whether this informal measuring of proof can be made more robust, and whether there is any value in that.
There was no evidence of impact on a range of longer term outcomes
The systematic review by Amanda McFadden and Kate Williams from the Queensland University of Technology found 19 studies on improving the research or evaluation capacity of teachers. Most of these were small qualitative studies that focused on teacher action research as a facilitator of professional development. There was almost no research that looked at the potential of building evaluation skills in teachers. The authors recommend that there should be more emphasis in teacher education on evaluation and evaluative thinking, given its potential to positively impact the professional practice of teachers and student learning outcomes.
One problem, though, is that, as the researchers point out, there is little evidence of the impact made by teachers carrying out their own evaluation or research.
They found some evidence that teachers who had conducted their own research had, for example, more positive attitudes to research, used systematic critical thinking skills in reflective practice, and that it enhanced their teacher identity. But there was no evidence of impact on a range of longer term outcomes, such as whether or not the teachers used more evidence-based practices or whether it resulted in more effective teaching and better outcomes for students. That’s not to say that it won’t, but at the moment there’s no research to say one way or the other.
Given that uncertainty, it’s perhaps not surprising that the idea that teachers might carry out their own research or evaluation isn’t mentioned in guidance such as the recent Early Career Framework. Until we have better evidence that a more structured approach to evaluation leads to better outcomes for students, it is unlikely that we will see such a policy widely supported in schools.
Over the last four years, the IEE has supported more than 25 teachers to run their own small-scale evaluation projects, and our experience reflects the findings of this review. The results of the individual studies have been interesting, and often surprising, though given that they are small-scale we can’t generalise the findings. The enthusiasm, knowledge and skills of the teachers and schools involved has increased dramatically, but it’s difficult to identify if there will be any long-term impact. Until we can identify the long-term benefit, such evaluations are likely to remain the preserve of enthusiastic teachers and schools, rather than a way in which the evidence base of the profession might be developed.
I think we should try to find out if it’s worth it, and it would take a fraction of the money spent on successive government initiatives. Though calls for better evaluation have often focused on student outcomes, evaluative approaches can be applied to any aspect of education. Changes made at local or national level may have all kinds of consequences, whether intended or unintended, and as the review points out “often have downstream effects on key issues such as teacher retention… the use of evaluation to zoom in on such mechanisms may be highly beneficial.”
Amanda McFadden, Kate E. Williams, Teachers as evaluators: Results from a systematic literature review, Studies in Educational Evaluation, Volume 64, (2020). https://doi.org/10.1016/j.stueduc.2019.100830