Three fire crews are currently in attendance at the West Bank Ferry Terminal site off Wherstead Road.
Pictures show dark smoke billowing over Ipswich from an industrial site near to the Waterfront.
Residents have also reported smelling a putrid odour in the air.
The police have also been made aware of the incident but have not been requested to attend the scene.
Suffolk Fire Service later confirmed that the fire was in a gravel washing plant machine on the terminal.
The service said on Twitter: “We will be remaining on scene and working with the Port Authority to make the area safe.”
Jodie Bowman, a nearby resident was watching the smoke from outside her home.
“One of the kids went round the back and said there’s smoke. Mum there’s a fire,” said Mrs Bowman.
“I phoned 999 and they were already aware of it.”
Mrs Bowman said it wasn’t the first time that her and her neighbours had seen smoke across the water in recent months.
A large fire broke out nearby at the former Tolly Cobbold brewery in February.
Mrs Fuller also lives nearby: “I was outside, I just saw the flames.
She also called the fire brigade.
“It smelled rubbery,” said Mrs Fuller.
A Suffolk Fire and Rescue Service spokesperson said: “Suffolk Fire and Rescue Service are currently attending a plant fire on the West Bank Terminal of Ipswich Marina.
“Potentially hazardous cylinders that were being housed at the plant have been removed and the electricity to the building has been isolated while crews fight the fire.”
The spokesman said that crews made progress tackling fire initially and stayed on site to dampen down hotspots.
The fire service also said that there were no indications anyone had been injured during the fire.
A stop was called on the fire at 1.03pm.
An ABP spokesman, said: “We can confirm that there was an incident involving a fire on Ipswich’s West Bank Terminal this morning, which has been successfully dealt with by Suffolk Fire and Rescue Service.
“An investigation will be conducted into the cause of the fire.”
Comments: Our rules
We want our comments to be a lively and valuable part of our community - a place where readers can debate and engage with the most important local issues. The ability to comment on our stories is a privilege, not a right, however, and that privilege may be withdrawn if it is abused or misused.
Please report any comments that break our rules.
Read the rules here