A lot of the problems are pretty similar to when gaming started to take off on mobile too. Initially we had mostly shitty ports/clones from other platforms and control schemes that largely sucked (virtual joystick anyone?). Over time quality has improved and control schemes have been figured out that work well with the touch screen interface to the point that mobile is a pretty good gaming platform (not for everyone but it certainly makes a lot of money).
VR will take some time to really find its footing but I wouldn't bet against it right now. Almost all the devs that I've talked to or watched interviews with are giddy with excitement over working with VR so I think the content will come, and once quality content arrives that will drive the adoption of VR gear by gamers too.