In California, and I assume the other states, there IS a prohibition of recording in public, if there's an expectation of privacy. If you're in public and yelling at the top of your lungs- there's no expectation of privacy there unless you're in a pretty deserted area. But if you're in public and having a normal conversation with someone, there generally is an expectation of privacy- you're far enough from others, and the others are moving around, that you expect your conversation by and large to be between you and the person you're speaking with. So it's not about if it's in public or private, it's usually in my experience about whether or not there's an expectation of privacy. Here's the relevant part of California Law (632PC):
"(c) For the purposes of this section, âoeconfidential communicationâ means any communication carried on in circumstances as may reasonably indicate that any party to the communication desires it to be confined to the parties thereto, but excludes a communication made in a public gathering or in any legislative, judicial, executive, or administrative proceeding open to the public, or in any other circumstance in which the parties to the communication may reasonably expect that the communication may be overheard or recorded."