Bucks Council has published proposals seeking to turn the old Bucks Sports and Social Club into a housing development.
Concerned residents have started discussions with their neighbours and council officials to seek an alternative.
Citizens in the Stoke Mandeville area feel starved of green spaces to play sports in and go to with their family.
Therefore, campaigners believe although the council has housing targets linked to the ambitious Vale of Aylesbury Local Plan (VALP), that other areas should be earmarked for future housing projects.
Another source of frustration for the people involved in the campaign was the decision to close the fields at the old club, fencing them off from the public at a time when residents were encouraged to exercise independently during lockdowns and social-distancing mandates.
A Facebook group has been created called, Save Bucks Sports and Social Club, organisers say during door-knocking sessions hundreds of residents have responded positively to their mission statement.
The group wants an extension on the current proposal period so it can come up with a feasible plan to take over the area and run it as a social club.
Members are also putting up posters and banners in neighbourhoods to ensure locals are aware of their plans.
The council’s proposal for the site on Lower Road involves building approximately 100 new homes, creating a substantial green space, a new link road, a new sports facility, and associated landscaping and parking.
At least 5% of the houses will be affordable, the local authority states, the new homes will be a mix of houses and apartments.
A council spokesman said: “The large island of green open space compliments the design principles of the scheme as a whole and will provide the local residents and wider community with an open leisure area to enjoy.
"This land will provide space for informal recreation and play, while also enhancing the biodiversity of the site and provide areas for wildlife.”
At a later stage plans will reveal how many of the houses will be reserved for key workers at the nearby hospital.
In an open letter sent to Bucks and Stoke Mandeville Parish Council representatives, campaigner Mungo Duncan said: “The Stoke Mandeville area has already been affected by five major house building projects.
"The local countryside has been put almost entirely off-limits by HS2 works where almost every field has been fenced off, and one of the last remaining open field areas will become a building site as a major link road is built, initially joining Lower Road with Wendover Road but longer term it may extend as far as Stone to join the Oxford Road.
"There is almost nowhere for people to relax or for children to play in this area, and Eskdale Park is simply too far away from the Stoke Leys area in most respects.
"Children and youths used to play ad-hoc cricket, football and other sports on the sports field area almost daily, people jogged around the perimeter every day.
"Families would have picnics or simply walk around the area. Some would use it for yoga classes, at night it was occasionally used for stargazing since there were few lights visible.
"Local schools used it regularly for nature rambles. It was an utterly invaluable much used & loved local resource, a source of pleasure, rest and education, one which has been badly missed by the whole community since the local authority - without any notice - barricaded it up more securely than any prison.
"It is certainly currently unused, but only because of the massive iron bars placed by the authority across any actual or possible entrance.
"During the pandemic we were encouraged to take exercise and yet Bucks Council chose instead to withdraw one of the main local facilities available for doing so.
"The various councils involved have made much in the press about Stoke Mandeville becoming a garden village and the old club site remaining as a key green space as part of the plan, but it is now entirely clear that this has never been Bucks Council’s intention.”
Mungo is keen to point out that residents aren’t looking to solely oppose the council’s plans, but want to work with the local authority to come up with an alternative solution.
He told The Bucks Herald: “We want to retain the green space completely, not just a small piece of it, but actually retain the whole thing.
"There has been a couple of recent drop-in sessions organised by the council.
"And the push back was significant. I think that was unexpected. That the strength of feeling among people in Stoke Leys and the Stoke Mandeville area is so great, that people want to retain it.”
Previous attempts by some of the parties involved to take over the field, since the club was closed were thrown out by the council.
Mungo recalls being told plans to run the social club, were thrown out for being ‘aspirational’ five years ago.
A Bucks Council spokesman told The Bucks Herald: “We are asking for views about an outline planning application.
“Our outline plans are for up to 100 new homes with a high proportion of affordable housing, a new link road between Booker Park School and Lower Road, and new sports community green areas.
“Using land which the council owns offers the opportunity to provide a higher proportion of affordable and key worker housing than is set out statutory requirements as well as the possibility of building homes for vulnerable adults and children.
“As part of the planning process we are carrying out extensive public consultation which is running until 9 May. Once the consultation closes, we will then review all the comments we receive before making a decision about the next steps.”