Investors buy rental properties to generate a passive income (i.e. get money in exchange for nothing), by exploiting an advantaged capital position (i.e. just by having more money to begin with).
Have you ever been a landlord? I have, and I guarantee that it was not a passive income. I had to arrange the financing in the first place, arrange for initial repairs and ongoing maintenance, find and coordinate with renters etc. And in my case, my renters actually were earning more than I did, they just weren't ready to buy yet. For them they got a nice place with no long term worries or commitments, at a fair price. And most importantly, they had a place to live.
That ability to benefit financially from housing you don't need for, you know, actually housing yourself, attracts people with money to spare into buying up more investment housing, increasing the demand and thus market price for housing.
Aside from the fact that a lot of people don't want to actually own their home at a given time for a variety of reasons, I don't buy this argument. A landlord can only benefit financially to the extent that people are able and willing to pay their rents - if you have a house sitting empty for a couple of months you are going to be losing money that year. In crunch markets, what people really are unhappy about is that they want to live somewhere where lots of other people want to also, and that means that prices are going to go up unless the housing stock can expand. In SF & NYC, that's not really the case, so you have rising prices. This is going to happen regardless of investors, because that's just how the market works. Investors don't cause the demand, though they may see it and hope to profit from it. See all the "investors" who got burned in Las Vegas when the bubble burst.