A good point. The processing to add "OPC" to the images on the fly might make your glasses rather warm. Also, the optics problem gets more complicated, because you do still have the increased dispersion due to the small spots. The wafers don't move and refocus the way a human eye does.
One advantage might be that you can dynamically change the effective distance to the virtual screen -- it's important people refocus to different differences fairly often. It seems to me there are "re-focusing" algorithms for images, and this could be a version of that (especially since we have the "true") image.
(I would mod you up, but I can't mod in a thread I'm contributing to ...)
dsojourner