It is 5.30am on the day I am meeting Toby Young, the gonzo journalist turned right-wing columnist turned free school founder, and my cat is making life difficult. He’s mewing loudly. I decide he must be dying of dehydration and get out of bed to find out what’s wrong.

Nothing is wrong. He has plentiful water and food. He just wants attention, and fresher food.

Young is not dissimilar. In 2009 he was living in London with four young children and a writing career that paid enough for fine living, but not for four places at fine private schools. This shouldn’t have mattered. By then, the capital’s education system was already on its way to being the best in the world. But Young, like my cat, wanted more than had already been served.

“I now know how hard teachers work and what a difficult job it is”

Before the 2010 general election he slagged off the state school system via every medium open to him and demanded the upcoming Conservative government to allow him – and other parents “like him” – to open the sort of school they wanted.

At the time I was a teacher in the sort of school Young would lambast. Schools like ours didn’t teach Latin. Schools like ours had low expectations. He, on the other hand, knew the source of all brilliance. He would sort it out. He, Mr Journalist Man, would be the saviour of mediocrity.

Like the cat, he didn’t just want something new, he craved attention too.

West London free school is not the first school you see when walking from Ravenscourt Park tube station. That sight goes to the beautiful gates of Latymer Upper – a private school founded in 1624 that costs more than £18,000 a year to attend.

The school that Young, with a troupe of other founders, pushed to open is some way right of Latymer, hidden behind a white wall and flanked by a high-rise Premier Inn.

The elegant building behind the wall, Palingswick House, was built as a school in the 19th century. But it was a slow, fraught and expensive business to acquire it as a finalised permanent site – at a cost of £5.8 million (before the refurb) for a school with just 600 places. Pupils were scattered across several sites while the work was completed.

The group now also operate three primary schools, on separate sites, and a sixth form is due to open in September.

Young isn’t there when I arrive, but I meet Hywel Jones, the soft-spoken headteacher, who is charming, intelligent, and down-to-earth. Appointed in 2014 as the third head in three years, he said at the time he was not daunted by leading a school that was a “national goldfish bowl”. As we wander around he gives the impression that such tours happen a lot.

We are joined by Young, who says he manages to go on “learning walks” about four times a term. “Though we won’t go into any classrooms with my daughter in, she finds it embarrassing.”

In a music lesson pupils are diligently learning keyboard and recording their progress on iPads. Year 11 is practising GCSE maths papers. In a science cover lesson, pupils are answering questions out of a textbook.

It’s all very normal. Nice normal. Calm normal. But, well … just normal.

I spot a poster about a barn dance later in the day and am encouraged to attend. It has been planned by the music department, which is particularly effusive about the school. Ten per cent of pupils are selected for musical aptitude (Jones tells me that the test is “not game-able”) and each receives dedicated musical tuition on a scholarship reminiscent of Oxbridge. I am also told all pupils study classical languages: another subject for which Oxbridge has lots of places, and few applicants. If many pupils go to those universities in a few years’ time, it won’t be an accident.

There’s a café across the road. We go in, but Young thinks it’s too loud. So he takes me to a coffee shop in a nearby garden centre. With trains rushing overhead. And crying babies.

We talk about the irony of our meeting. He, the journalist, is now an academy trust chief executive. Me, the one-time teacher, is now the interviewer.

I ask what motivated him to continue with the school in the face of adversity – including fights over the buildings and his two lost headteachers.

“I believed very strongly in the concept of setting up a school that could provide a knowledge-based education to children of all backgrounds regardless of abilities, and was confident enough in the strength of that concept not to be dissuaded by the opposition.

“I had four children who at the time were all at primary school. There was the nagging worry about where they would go to secondary school if we didn’t get a free school open.”

Why not send them to private school? “I couldn’t afford it… Also I didn’t go to private school myself, and seeing my friends – some of them – who had sent their children to one, they seemed to care too much about things like whether or not their child got an invite to a Russian billionaire tycoon’s daughter’s 16th birthday party.

“Their children seemed to grow up very quickly and absorb lots of the plutocratic values that I was exposed to when I worked at Vanity Fair [in New York] in the mid-90s and have a visceral dislike of.

“And, I suppose, I really want my children to be educated alongside children from all parts of the local community and not a tiny elite.”

Why not walk away once it opened, though? Why become the chief executive?

“I did think at the beginning that I would bow out once the school opened.” He pauses. “I suppose, having spent two years working very hard to get the school and having persuaded local parents to place their faith in you and your colleagues, you can’t really walk away. You have an overwhelming sense of responsibility and that grows each year as you get another 120 kids. I think the fact it didn’t have a permanent site made me reluctant to take a back seat as well.”

He sips his flat white coffee. “Having said that I am stepping down at the end of this academic year.”

An advert for his replacement went out the day before we meet. The new CEO will be full-time, (Young does three days) and, unlike in 2013 – when the role was last advertised but failed to turn up anyone suitable – more money is available.

What sort of person is he looking for? “Our multi-academy trust needs to grow if it is going to become sustainable over the long term. The Department for Education’s research about multi-academy trusts suggests that anything less than eight to ten schools is quite hard to sustain. What we need, if we are going to grow from four schools to eight to ten schools over the next five years, is someone with the experience and skills to grow an organisation like ours. So, ideally, an ex-headteacher or someone who has already run a medium-sized academy trust or someone who has experienced a growing comparable business.”

Does he have someone in mind? “No.” Does he believe they exist? “I hope so.”

Back in 2011, just before the first school opened, he told an interviewer that he planned to open 25 schools, and develop a private company to advise on the opening of schools. One of the reasons why Young recently suggested a new college for schools leadership training, which Sir Michael Wilshaw has urged the government to find funds for, is that he now realises there is a dearth of people who can make 25 schools suddenly appear and run them effectively.

“The sorts of people who I think, I hope, the course at the college would appeal to are senior managers of West End theatres or of NHS trusts. Having worked in a theatre company would be particularly good.”

This sounds an outlandish attention-grabbing statement. I prod further. He prods back.

“I am always struck with the similarity between what actors do and what teachers do. Teachers are kind of on stage and are, at some level, having to perform and having to carry off a performance. The skills aren’t dissimilar and some of the strain that puts on people is the same – it’s exhausting having to perform for four or five hours a day, sometimes more. It’s like having to do a matinee and an evening performance of a West End show every day. So, someone who is used to managing a troupe of actors, and dealing with the stresses and strains they are under, would be quite well suited in some ways to being a senior leader in a school.”

“I am prepared to vote with my children”

It is a poignant and thoughtful answer. Wondering if his attention-grabbing habit has waned I ask if, given what he now knows about the difficulties of leading schools, he regrets his disparaging remarks around the time of the 2010 general election.

“Yes. There are a lot of things I regret,” he says. “I was very critical of England’s public education system under the last Labour government, and I hadn’t grasped how difficult it is to do better, and to bring about system-wide improvement.

“The last government and this government have achieved a remarkable amount, and I do think the direction of travel is the right direction, but there is no question that it was arrogant of me to believe that just having high expectations and believing in the benefits of a knowledge-based education for all, that those things alone would be enough to create successful schools.”

Did he really think back in 2010 those things would solve everything, or was that just media bluster?

“As someone coming into education from the outside, the bits you see of other schools are only the tip of the iceberg. You’re not aware of everything that is going on beneath the surface. You think, ‘well, I could do better than that’, as you are pointing to the tip of the iceberg, without realising how much more there is to it.”

He sighs. “If I could rewind six years, and know then what I know now, I would have been much less critical of other schools, local authorities, and England’s public education system in general.”

But, he can’t go back. So what of celebrities who follow in his path? In the US, Puff Daddy, the hip-hop artist, is advocating for a new charter school [the US equivalent of a free school]. What’s his best advice?

“Try to fly under the radar and not make any public statements about education, but support the people you are working with financially and emotionally.”

I begin another question. He interrupts.

“Can I just say … one thing I really regret is that I gave a quick interview in an ITV programme about teachers in which I was quite dismissive about workload complaints. I regret that, enormously. I now know how hard teachers work and what a difficult job it is.”

He looks pained.

I ask what he’s proud of. He talks at length about his free school “team” – the heads of the schools across the trust (not just Jones, but also the primary leaders), the teachers, the pupils, the other founders. He insists the sustainable embodiment of a vision dreamed up not just by him, has been done by the group. He names several of them. He is very concerned that it doesn’t appear in the interview that he did this all alone.

But he admits that there were moments when it felt “quite fragile. For it not to be fragile any more, but in rude good health, is a source of pride.”

He looks plaintive. As if there’s more to be said. Does he wish he’d been asked another question? “Yes,” he says, “it’s that… not only is my daughter in year 8, but my son is going into the secondary in September – and I think it’s quite important that people involved in running schools should, if possible, send their kids to those schools.

“It has always struck me as scandalous that successive generations of secretaries of state and education ministers went private when they were responsible for the public education system, and too many governors and heads don’t send their children to the schools they work at, or opt-out of the state sector.

“I am prepared to vote with my children. Not because I feel I have to, but because I now genuinely think this is one of the best secondary schools in the area.”

He finishes his coffee. And finally looks happy to wind up the interview.

The next day I get an email asking if I made it to the barn dance. I tell him that I didn’t but that I had let Jones know. We agree the head would make a great future profile interview of his own.

A few days later, I discover the cat has eaten a roast chicken left to cool. He slinks away when caught, looking guilty for taking what wasn’t his. Given half a chance, I suspect he’d do the same again.

It’s a personal thing

What is your favourite book?

David Copperfield. It is just such a wonderful story, it is so moving and the characters are so vivid. I read Nicholas Nickleby first and thought that David Copperfield was a polished, better version. No other book has given me more pleasure.

If you could be any animal, what would you be?

I would be a Hungarian vizsla puppy. They are very pretty and quite big. We have just got one and my wife dotes on him, as do all my children. I would like to be him so I could soak up all that love and attention.

If you were invisible for a day, what would you do?

I’d go on a series of learning walks across all our schools. The problem with learning walks is that Heisenberg’s uncertainty principle applies; you are never sure if it is really like that when you’re not there. If you were invisible, you could take the uncertainty away. But I don’t think it would be different!

If you could live in any period in history, which would you pick?

I would quite like to live in ancient Rome. But of course when one thinks about that one imagines one would at the very least be a senator . . . if I were a slave I imagine it would not be much fun. It’s like when people say they want to live in the 18th century without realising that, as one historian pointed out, people in the 18th century were in pain half the time.