So, I'm in their system, despite being really well known as the paranoid "they're out to get me" guy to pretty much everyone who knows me.
And this is why privacy/data protection laws need to be updated to have far more teeth than they have today. When you have an organisation as influential as Facebook and it is actively encouraging other people to do things like providing your picture or your phone number with or without your knowledge or consent, any argument that some use of that data about you is permitted under their ToS has no weight if you're not a Facebook user yourself, but it seems clear that they're storing the data anyway. Actually, I'm not sure how that's not already illegal, at least within the EU, but the regulators don't seem in any hurry to take action and even if they do the penalties are little more than the change in Zuckerberg's pocket.
FWIW, I am similar to you, being well known among my friends as someone who doesn't want to share his personal details with Facebook. I feel sufficiently strongly about this that in the situation you described I would have made it very clear to my "friend" and his wife that I would no longer consider them friends if they thought it was funny to violate my privacy in that way, but then again I'm also confident that I would never have to go that far with anyone I consider a friend in the first place. I'm sorry if you're not always in such a happy position with the company you keep.