Mr Smith is opposing plans for a third prison at Grendon Underwood because it is on a greenfield site and the government has stated it is seeking brownfield sites for development.
Speaking in the Commons on Tuesday, December 7, he said: "Plans for a 1,400-capacity new prison in Buckinghamshire was met with a wall of opposition, thousands of objections, particularly around the loss of greenfield sties, open countryside and agricultural land.
"So, with other parts of government talking about a brownfield preference, will my Hon Friend lock in to this strategy that all new prisons should be built on brownfield and brownfield only land?"
Prisons Minister Victoria Atkins replied: "My Honorable Friend is a staunch defender for his constituents and is assiduous in advocating their concerns in relation to this and other matters.
"I can't comment further on the particular project because it is in the process of planning but I am very happy to meet him to discuss this further because I am sensitive to the concerns that he has raised."
Afterwards, Mr Smith said: "The government this week announced the Prisons Strategy White Paper, setting out a plan for safer, more secure prisons that cut reoffending as we build back safer.
"There is much-to-be-welcomed in the White Paper, particularly in terms of cutting reoffending.
"Prisons keep people safe by taking dangerous criminals off our streets, but they can only bring down crime and keep the public safer in the longer term if they properly reform and rehabilitate offenders.
"The Prisons Strategy White Paper sets out the government's plan for providing prisoners with rehabilitative drug treatment, deliver increased prisoner education, support prisoners into jobs as they leave custody, and implement airport-style security to reduce the flow of illicit contraband in prisons.
"However, as part of the drive to increase prison spaces, and our unhappy experience in Buckinghamshire with proposals for an inappropriately sited 1,400 capacity new prison adjacent to HMP Springhill and HMP Grendon, I challenged the minister after her statement to lock in that all new prisons must be built on brownfield and brownfield only land."
Buckinghamshire Council announced this week that it will be looking to see what brownfield sites there are that may be suitable for development or other purposes, as opposed to greenfield sites.