I'm beginning to feel we are entering the endgame of Boris Johnson's Premiership as Tories from all levels of the party in Suffolk are now finding it increasingly difficult to come to his defence in the face of the latest 'partygate' allegations.

The latest revelations about the "Bring your own booze" party in the Downing Street garden, on the day that ordinary members of the public were being told they could only meet one person from outside their immediate household in a park, appear to be the straw that could break the camel's back.

It would be wrong to underestimate the level of affection there still is within the party for a man who led them to their largest election victory since 1987 - and who helped them win seats which had not been held by the Conservatives for decades.

However, an increasing number of formerly loyal supporters are now privately expressing growing exasperation at the behaviour of the Prime Minister and staff in Downing Street - and the latest revelations have added to that.

I am now hearing from Tories who enthusiastically backed Boris' leadership bid in the summer of 2019 who now say that his total failure to understand the anger and hurt that his actions provoke is simply unacceptable.

I don't think his admissions in this week's Prime Minister's Questions that he attended the Downing Street party changes anything. It was clearly a party.

It was significant that not one Tory MP in Suffolk or North Essex was prepared to defend the PM's actions at the start of the week. Some either said we should wait for the results of the inquiry or said the allegations were very worrying.

Others - mainly ministers - were keeping their heads down and did not respond to our calls. From what I gather that's a pattern being played out around the country.

And it's not just MPs who seem to be losing patience with the shenanigans coming out of Number 10. I've heard from councillors and grassroot Tories who are getting thoroughly fed up with the PM's behaviour and now feel he is much more of a liability than an asset.

I've even heard people say he should just be ditched now to give someone else a chance to rebuild the party's reputation before the next election.

Mr Johnson's behaviour is now starting to jar with many in his party. He appeared to try to laugh off the early reports of the May 20 party in Downing Street.

"For someone who prides himself on being a 'man of the people' he seems totally out of touch with the concerns of ordinary voters," one Tory said to me.

"I can't believe that he wasn't at this party in May - so many people seem to have seen him there."

One of the more interesting comments I hear from his internal critics is that Mr Johnson isn't industrious enough to be Prime Minister - and within the party the comical performance at the CBI when he lost his place and started wittering on about Peppa Pig is still a big talking point.

A leading Tory said to me: "That was just appalling. It showed the world what we've come to realise over the last two years - he appears to be too lazy to read briefing notes and too arrogant to realise that people don't see this buffoonery as an asset for a Prime Minister."

But some of his party critics are very wary about mounting a challenge.

One said: "The problem is that if there was a vote of confidence but he still got about 60% of the MPs to support him, he would simply see that as an endorsement and carry on.

"I don't think there's any point in 'the men in the grey suits' telling him his time is up because he believes he's got huge support in the country. I don't think he is a great electoral asset any more - but he does."

Their one hope is that while the Prime Minister is not the asset he was, the Labour Party hasn't really convinced anyone it is ready for power. It doesn't look like the slick electoral machine Tony Blair and Alastair Campbell forged in the mid-1990s.

The Tories are worried that the new Labour front bench does look more credible than it has at any time since 2010. But do the electors really see it as a "government-in-waiting"?

As I said there are still Boris loyalists. You only have to look at social media to see that there are still many who will go online to say "no one is interested in this tittle-tattle."

If they really believe that then they are deluded. This is the talk of dinner tables across the country.