Research released last week on the Shanghai teacher exchange reveals that despite sending 60-plus teachers to and from China every year since 2014 – and supporting them to establish mastery practices in their schools – there has been no improvement to key stage 2 SATs in the first cohort of pupils.

The report was commissioned by the Department for Education on one of its own flagship programmes. So is this the nail in the coffin for Shanghai-style teaching for mastery, or could there be other less obvious benefits (apart from the government’s desire to boost education trade with China) to the teacher exchange?

“We’re not going to throw it out,” says Michelle Ginty, the headteacher of a two-class entry primary in northwest London, who sent one of her assistant heads. “It’s given huge energy to all the teachers – especially the support staff – who often work with the least able children. Our teaching assistants find it really powerful.”

Ginty remains hopeful that it will boost results at Salusbury Primary School. “It will impact significantly on KS2 outcomes for the year 5 cohort that participated,” she says. “It can’t not have an impact. Even if what you were observing was a disaster, you’d still be learning from it!”

The data says otherwise, however. Looking only at the schools that followed through and implemented a “substantial” mastery approach for two years, there was no significant increase in KS2 outcomes – something that baffles Mark Boylan, the lead evaluator. “Given the evidence for the effectiveness of different parts of the mastery approach, I would have expected a positive effect,” he says.

Ginty is not alone in her enthusiasm. Teachers across the country believe that the exchange programme, coordinated by the National Centre for Excellence in the Teaching of Mathematics (NCETM), is having a positive effect on attainment. “What this shows is that we’ve got to be careful about not allowing our strong beliefs make us selective about the evidence that we’re willing to pay attention to,” Boylan says.

But is this research the final word?

Yasmin Hussain, maths lead at Salusbury Primary School and Natalie Stevenson, primary mastery lead for London central and NW maths hub

One dissenting voice comes – unsurprisingly – from a non-profit that’s developed an alternative maths mastery method that it sells to schools, and whose independent Education Endowment Foundation trial showed a positive result.

“It takes time to change practice,” says Helen Drury, executive director of Mathematics Mastery, which grew out of the Ark academy trust and now works with about 500 primary and secondary schools.

“A child who’s been labelled as relatively poor at maths for five years, they’re going to take some convincing that success is a real possibility. It’s going to take more than two years.”

Charlie Stripp, the NCETM director, says he “found the headlines quoting ‘no impact in key stage 2’ strange because there was never any likelihood that there would be any impact on key stage 2 SATs results in the schools taking part in the early exchanges.”

The results for younger pupils were more promising, with pupils at schools that implemented a high level of mastery almost 1.5 times more likely to attain the expected key stage 1 level in maths.

The evaluation, it should be noted, was on the original teacher exchange, not on the broader set of activities that has since developed into the NCETM’s teaching for mastery programme – which is still “developmental”, Stripp says, with feedback from participant teachers “constantly used to refine the training”.

It’s not the effectiveness of the method that is in doubt, Drury says – it’s all about the implementation.

“The challenge isn’t working out the most effective way to teach maths. There’s an extremely strong evidence base for this approach to teaching maths. Much of it is from UK researchers.”

We’re not starting from ground zero

Where higher-performing areas such as Singapore or Shanghai have an advantage, she says, “is not that they know how to teach maths effectively and we don’t – it’s that they’ve invested a great deal in connecting that knowledge with classroom practice, so every teacher has got a really coherent multi-year programme of professional development underpinned by classroom materials that have been carefully crafted by expert practitioners.”

Are there dangers, then, in looking abroad for models of best practice? “I think it is unhelpful for the NCETM teaching for mastery approach to be seen as Shanghai-based,” Drury says. “Because you can’t import a teaching approach or borrow a pedagogy.”

Yasmin Hussain and Natalie Stevenson lead a training session for teachers in the London central and NW maths hub

Yasmin Hussain, maths lead at Salusbury, exudes an almost religious devotion to the method. She’s just led a training session for about 80 teachers in her maths hub, including a demonstration lesson by a visiting Chinese teacher.

“When you’re trying to work collaboratively with people and for people to be as passionate as you are, they need to see it,” she says. “It’s very, very powerful to watch a lesson, especially where it’s been very carefully crafted.”

Many of the answers to the teachers’ questions seem to be about determining what Shanghai teachers would do, rather than debating why a particular method is the right one to use. There’s a lot of back and forth, for example, trying to work out why the Shanghai teachers don’t use bar modelling – a common form of representation in English primaries – as if understanding that question might unlock some deeper wisdom.

This is one potential drawback of the “cascade” model that is central to the teacher exchange. Teachers receive mastery training from the NCETM, spend a week or two in Shanghai, and then become the local guru on teaching for mastery. How deep can their understanding be?

Hussain says her aim is not to import the system wholesale, but to expose teachers to a different way of doing things, and “get people to think about the journey of the maths during a lesson, to break it down”.

Natalie Stevenson, primary mastery lead for the London central and NW maths hub, adds: “The only people who think it might be mimicking something are those who haven’t been involved with it. There’s a recognition that we couldn’t replicate it, even if we thought it was a good thing.”

She’s not advocating that teachers in England throw out everything they know. “It’s not that we’re starting from ground zero, we’ve got some excellent practitioners here. The very experienced teachers will take lots from that but they don’t go, ‘I’m going to teach exactly like that tomorrow’.”

Our teaching assistants find it really powerful

Instead, it’s about learning the pedagogical principles, which she lists as: variation, representation, mathematical thinking, fluency, and coherence through the lesson – five principles formalised by Debbie Morgan, primary director at the NCETM, who – ironically enough – is herself often spoken of with something of a religious devotion among mastery enthusiasts.

It’s a week when schools all over England are hosting teachers from Shanghai, and inviting local schools to come and observe, as part of the NCETM’s teaching for mastery programme. The year 2 class I watch is sitting in the school hall, with rows of teachers sitting behind, observing.

The lesson starts simply and concepts are layered gradually. Familiar key terms and phrases are used repeatedly, and the pupils confidently use terms such as “factor” and “product”, which they have been taught earlier in the week. Individual children are regularly asked to answer questions, then this teacher folds her hands behind her back and leans forward, sweeping her head from side to side to include the whole class, as she asks, “Do you agree?”. Her movements are precise, fast, deliberate.

The same sum is approached from multiple angles – through addition, multiplication, subtraction, and using pictorial representations – a key element of this pedagogy. The idea is that pupils get a much deeper understanding of the mathematical operations, and the relationships between numbers – and fundamentally, everyone leaves the lesson understanding the two times-table.

“Teachers thought they understood what depth meant, until they observed the lesson,” Ginty says, describing the first time her teachers were exposed to one of these “showcase” experiences.

She is confident about the beneficial impact the method will have, especially on her lower-achieving pupils. “Children with SEN were able to access the lesson and perform at a similar level to their peers.”

In sum, most researchers agree that the mastery method works. But was flying a few hundred teachers half way round the globe an effective way to change practices in England?

Drury, who has good relations with NCETM, is diplomatic on that question. “Whilst we understand that looking to Shanghai is an exciting catalyst for change, we don’t put all teachers on planes!”

Boylan, on the other hand, gives the stock academic’s response: “It may not be what the government wants to hear, but it needs more research.

There are no quick fixes, no silver bullets.”