Old buildings will be demolished to make way for phase one of the Waterside North scheme, prompting a warning from MP David Lidington.
Under the plans Annex B, a building between Judges’ Lodgings and Walton Street will be demolished as well as buildings 5-7 Walton Street next to the White Swan pub.
Planners say that getting rid of the structures will open up the square, and better showcase the historic Judges’ Lodgings.
But MP David Lidington urged caution when considering whether to demolish any town centre structures, and said how vital it was that residents made their views heard.
He said: “Whether we should be removing relatively old buildings is up for debate.
“We made lots of ill-advised but well meaning decisions in the 1960s and 1970s and it is very important that people have their say.
“But there is no doubt that we need to improve Aylesbury’s offer as a retail centre, it is important and urgent and we have been losing out.
“I hate Hemel town centre, but when you look at the shopping malls there they actually have a better range of shops then central Aylesbury has.”
The new square will contain restaurants, apartments and a public space which planners say could be used for live music and buskers.
Walton Street will be made more pedestrian friendly with the road possibly being raised to pavement-level to make it a shared space.Disabled parking and loading bays will remain.
This week, many visitors to the public exhibition on the scheme gave it warm reception.
But some suggested key points including focussing on smartening up the existing town centre first and bringing more leisure and recreation facilities into the town.
Other buildings will also be demolished as part of the scheme to make way for a new 98 space car park.
Rear wings of the county offices, the Police Superintendant’s House and the old police station in Exchange Street will go to make way for the development.