How can ‘quick-fire’ research help us answer timely questions?

The Education Endowment Foundation (EEF) is probably best-known for commissioning independent evaluations of large-scale randomised controlled trials (RCTs). These are widely recognised as one of the best and least biased research methods for finding out what impact a specific programme or approach can have on attainment. Used in conjunction with information on implementation and cost, RCTs can give us powerful information about whether a programme is likely to work in a particular context.

One drawback is that large-scale RCTs take considerable time to deliver and evaluate. The typical EEF-funded RCT takes three to four years from commission to publication. Sometimes, we a quicker approach.

That’s why we commissioned a series of ‘nimble’ RCTs as part of the first year of delivering the national tutoring programme (NTP). These test elements of a programme instead of the whole thing. Within NTP, we identified the importance of pupil attendance at tutoring sessions and felt there were opportunities to rapidly learn more about how to improve pupil engagement.

The Behavioural Insights Team (BIT) developed and tested three different strategies to find out if ‘light-touch’ interventions could boost attendance at NTP tutoring sessions during the spring and summer terms of 2021. These interventions completed in less than 18 months and the findings, published this week, provide interesting and important insights for schools and tutoring organisations looking to engage pupils in tutoring interventions.

A small number of tuition partners took part in the trials of the three strategies, all of which were based on insights from behavioural science:

• Engagement-boosting reminders, where behaviourally informed reminder messages were sent directly to pupils via email.
• Prioritising tutoring relationships, where tutors completed a short web-based activity focused on relationship-building strategies that could be used with pupils and received reminders about the personal strategy they developed in the activity.
• Snap surveys, where pupils and tutors answered quick-fire questions about their interests, hobbies and values to identify similarities that could help them build a positive relationship.

The first two approaches performed no better or worse than the control groups, with whom tuition partners used ‘business-as-usual’ strategies to encourage attendance. However, the approach that aimed to leverage similarities between pupils and their tutors did improve attendance rates.

Four of the 33 tuition partners took part in the ‘snap survey’ trial. Once they’d completed the surveys, tutors and pupils received instant feedback on their similarities. Tutors also received reminders of their similarities with their pupils for the next five weeks, including some suggested conversation prompts. Tutors then used teaching strategies that incorporated their pupils’ interests to help build a positive relationship.

Tutors were randomly assigned to either receive the snap survey or business-as-usual relationship-building activities. The evaluation found that pupils of tutors who received the survey had higher attendance rates than those of tutors in the control group. The same was also true for pupil premium pupils, among whom this activity also led to increased attendance.

Interviews with tutors conducted as part of the evaluation suggested that the intervention may have been particularly effective for tutors who had less experience. Tutors also reported that the survey contributed to the pupil-tutor relationship by stimulating conversation between pupils and tutors which can be difficult, particularly in an online environment. Some tutors perceived the activity to work particularly well for specific groups of pupils, for example, those with SEND, who may find it harder to engage with teachers and tutors or in unfamiliar contexts.

This is a significant finding, particularly as the trials were conducted during a period of widespread disruption to schools, including the school closures that spring term. The findings will allow for purposeful action and improvement to this valuable programme. Armed with this new knowledge, tutoring organisations will know to prioritise and capitalise on commonalities between tutors and their pupils to secure and sustain pupil engagement.

And by highlighting the value that different research methods can bring in helping us to answer meaningful questions about programme implementations quickly and robustly, these reports don’t just affect tutoring practice, but the very way we conduct research.