Tom,
 
A lot depends on your neighbors.  You can do some things with your bees but people's perceptions about bees are another matter.  I work for the local agriculture department (or "bee police" ala The Olde Drone) and since we don't inspect bees in this county anymore, most of my time spent on bees is dealing with swarms calls, complaints about neighbor's bees, bees in the old shed and so on.  About half the people who call claim to be allergic to bees and fear death if stung.  I suspect only a fraction of them really are truly allergic and at risk of anaphylactic shock from a sting or two.  The danger of getting stung is quite low.  I occasionally have a hive in my backyard.  I have one there now, a swarm I got last week.  I mow right around them with a power lawnmower.  My neighbors do not complain.  They probably don't know they are there.  I did put a wooden lattice on part of the fence so they have to fly up when passing through the closest neighbor's yard.  
 
Bee poop is sometimes the problem.   I had a call last year from a neighbor a block and a half away.  Every car and house for 200 yards had little yellow flecks on it.  That guy had 4 or 5 vigorous hives in his backyard.  I convinced him to cut it back a bit.   Now he has two and I am not getting any complaints.
 
I heard an anecdotal tale about a beekeeper who had some hives on his roof, next to a car dealer.  The bees would defecate all over the new cars.  A lawsuit was mentioned but they reached an agreement and paid him to remove the hives.  
 
One beekeeper in another town had the neighbors go to the city council and complain.  The city passed a law against beekeeping.  This is what I really want to avoid.  But then the neighbors moved and nobody is enforcing the law, so he has snuck a few bees back in.
 
Last week I had a call from a new resident.  Her neighbor had bees and she was of course highly allergic.  She said the bees were swarming all over her car.  I went to see the site and found a healthy colony just across the fence and a freshly-caught swarm in a nuc in the front.  The beekeeper was fixing his car in the drive.  I knew him, he had about 350 colonies but the river took them two years ago.  Now he is a gardener.  He agreed to move the bees but needed to keep a nuc around because he was providing apitherapy for some people.  His neighbor seemed to understand and accept the situation. 
 
My advice is to keep them out of sight in a fenced area.  Higher fences make better neighbors.  Don't have too many colonies.  Give the neighbors some honey if they figure out you have bees.  It also helps to educate the neighbors about the value of bees, what a swarm is about and how not to get stung around bees.  Of course if any bee stings them, they will be sure it is one of yours.
 
Bob Roach