if we'd hovered over his house to ogle his teenage daughter
I never read anything that indicated the drone was 'ogling' the daughter.
I have two UAVs and use them for real estate aerial photography and to take pictures of people's houses for them (It's something that people do up here in Nova Scotia). I have to sometimes fly over other people property to get the correct viewpoint or angle and I only need permission to take off or land on somebody's property. If they have an issue with anything I'm doing they have the right to contact the RCMP. This has happened to me twice. On both occasions I showed the officer my documentation and insurance and the matter was dropped.
I think that people should be required to register their UAVs in the States, it will go a long way in ensuring that people understand what they can and can't do with their UAVs. People should also STOP calling them 'drones' because of the negative connotations with that word, it's instantly associated with spying which is NOT a good thing.