The fire that devastated the former Tolly Cobbold Brewery on Sunday has shocked the town – and prompted an outpouring of sadness.

While it would be wrong to comment on the causes of the blaze while police and fire investigations continue, there are rightly serious concerns about how such a prominent building was allowed to sit empty and decay for nearly two decades before the fire broke out.

And that looks like a very sad tale of a building that seems to have suffered from being out of sight, out of mind for many people.

I also question whether it has been ill-served by a over-prescriptive planning service which effectively prevented any development and has resulted in it reaching its derelict state.

Ask most people in Ipswich what they think of the Cliff Quay Brewery and they will say it's a lovely old building - but how many people actually see it in all its glory on an everyday basis? Only a very small number.

To actually see the building in all its glory you have to be on the Ipswich Port Authority premises at Cliff Quay - and ABP which owns that quite understandably wants to keep the general public away from the working port.

I know you can see part of the building from the bottom of Landseer Road and Cliff Lane, but you don't get any sense of its grandeur and its importance to the area until you are standing on Cliff Quay - and the port authorities do not allow the public to do that.

The effect of that - as Paul Clement from Ipswich Central said this week - is that while most people in Ipswich will say they value the brewery building, it's too peripheral to the town to make it really important to them . . . until there's a major incident there like the fire.

I know exactly what he means. As a building the brewery is far more important to the history of Ipswich than the old Grimwades store or the derelict sites at the Stoke Bridge entrance to the Waterfront - but because of their position they occupy a far higher profile in discussions about the town's development.

Another fear I have about the future of the old brewery is that strict planning rules may have helped contribute to its lack of development since it closed in 2002.

There are several different strands here. In the first place it could not be converted into flats, which appeared to be the obvious future for it, because it was too near the former oil storage facilities - residents could be at risk if there was a fire or explosion there.

Then there were problems with converting the building because it was listed inside and out. That meant any conversion to any other use had to take account of the internal layout and maintain most of the existing features.

That, in turn, meant any conversion would have to be limited and very expensive - and being on the edge of the town with no opportunity to exploit the brewery's position on Cliff Quay several different proposals failed to attract the finance needed.

Today hopes of finding a new use for the building seem further away than ever - and I am increasingly pessimistic about its future.

The day after the fire I managed to track down the name of the company that owns the building and the name of its only director. When I contacted his address and asked to be put through to him, the person I was connected to told me they knew nothing about any building in Ipswich and the conversation ended abruptly.

That does not fill me with confidence that the future of the building is secure.

In fact, I have no confidence about what will happen to the former brewery and its associated buildings - the old Brewery Tap pub and the Brewery Cottage which are attractive buildings in their own right.

As of now, the authorities need to speak to the owner of these buildings as a matter of urgency and then try to find a way of improving the area. But that won't be easy. The borough could, in theory try to use compulsory purchase orders to take control of the site. But that could be very expensive.

And then what would happen? The council does not have unlimited funds to finance an expensive conversion of a building it has no immediate use for.

The nightmare is that this derelict building that was once such an attractive feature of the maritime entrance to Ipswich will just be allowed to decay so ultimately the only solution will be to send in the bulldozers.