The Longhurst Group, a housing association, wants to buy the land from The Croft's owners the Cook family, to restore the house to its former glory and build up to 60 bungalows in the grounds. They would be for people over 55.
Members of the public were invited to see the proposals at a consultation event on Wednesday.
Alan Whyte, partnerships director at Longhurst Group, said: "Local residents have been very concerned about certain aspects of the site being developed.
"We want to listen to their views and then go away and revisit the plans to see what best accomodates the people's views.
"We are not prepared to go to public appeal, as applications have done in the past.
"We want to get it right the first time.
"The Croft itself appears to be integral to people's concerns and that needs to be a main focal point and people living immediately around the site are very concerned about issues such as traffic and that is something we understand and want to address."
The mansion house would be restored and its main use would be as a community lounge for residents. It would also have a manager's office, small kitchen and possibly suites for guests.
The driveway leading up to the house would be unchanged.
The homes would be aimed at over 55s, with many of the residents over 70 and Mr Whyte estimates only around 30 per cent would own a car.
A third of the housing available would be affordable homes for rent or shared ownership. There would be shrubs and trees around the housing and possibly a bowling green.
Bourne Preservation Society Jack Slater was at the event.
He said: "It was very positive and the driveway and look of The Croft remains the same."
Town councillor Helen Powell said: "I was worried that it was going to be two-storey buildings, but it is bungalows so I am pleased about that."
Mayor of Bourne, Shirley Cliffe said: "They are doing the right thing by consulting people and I am quite impressed with the plans. I am definitely in favour of what they are suggesting."
Connie Glover of Wood View, Bourne, attended with her hus-band Derek.
She said: "I just want to see The Croft tidying up."
A planning application has yet to be submitted and Mr Whyte said he didn't envisage bungalows being available on the site for at least two years.
People packed Bourne Corn Exchange for a planning inquiry in 2005 to object to plans to demolish the house and build 51 new homes on the land. The planning inspector rejected the proposals.