I'm not sure how long it's been since you last tried to park in San Francisco, but about 5 years ago the city started a program called "SFpark". Basically it's a system where parking meter prices are set dynamically, based on demand. I think the goal is to have the smart meters on each block set prices just high enough so that there is one free space on that block. There's an app that lets you check how much it costs to park in a given place, and there's a cap on the cost.
So if you're parking in an SFpark area, it should virtually always be possible to find a spot, if you need it badly enough to pay the price (up to ~$6/hour).
The land that a parking spot in SF occupies is worth more than the typical car that is parked on it. I imagine it's the same in New York. It's crazy that a society would give this away for free - we don't expect free cars, but for some reason many (most?) people feel entitled to free parking.