A few thousand units is not nearly enough to move the needle on price. Setup costs diminish greatly at around 10,000 units (usually) but that's isn't where the big money is here
They have shipped well over 20'000 developer kits so far. That's a device only sold through their website, known to be low-resolution, lacking position tracking and to become obsolete within a year due to release of the much improved consumer version. I have little doubt that they will sell a lot more units once the consumer versions hits the retail shelves. They also have something like $90 million venture capital, so they certainly can do some volume ordering.
The primary reason is that this technology always has been a solution looking for a problem.
Total immersion is a problem and people have wanted to solve for a long long time. At the moment it's still a little niche due to lack of hardware, but there is no shortage on people with tipple monitor setups, TrackIR and other gadgets to get as close to the real thing as possible. Rift can provide a better experience for much cheaper.
No matter how good the headset is, there simply isn't any evidence that there is mass market levels of demand for full immersion VR in any of the likely markets.
Look at some reaction videos on Youtube, everybody from 6 year old kids to 90 year old grandmas seems to enjoy the experience, a lot. Disney also did some research back in the 90's with they Aladdin Virtual Reality Carpet Ride and concluded that it's basically fun for all ages, no need to be some hardcore sci-fi geek to enjoy a bit of virtual reality. All the gaming aside, who would say no to an IMAX cinema in his living room? Virtual reality can provide that.
It's kind of like a Segway - neat but really just an expensive toy with limited real world application.
The Segway cost as much as a small car, so it's not that surprising that it didn't take of. If the Segway would have cost as much as a bike it would have had a much better time with mass market adoption. The Rift by contrast is pretty cheap, well within the realm of other gaming peripherals, cheaper then a big TV.