Second, if h264 needs to be licensed at such exorbitant prices, how do x264, VLC, and MPC-HC do it?
I don't know about MPC-HC, but x264 (which is the h264-compatible codec used in VLC) is a cleanroom reverse-engineering job. So, it's technically not h264. Just very compatible. And it's GPL'd. However, we now have the question of why the Mozilla folks aren't supporting a GPL'd codec. I'm guessing because there are possible legal patent-related issues surrounding a reverse-engineered codec like x264.
Doing a bit more searching, we find this:
http://www.unmediated.org/archives/2005/05/videolan_x264_e.php
x264 is a free library for encoding H.264/AVC video streams. x246 has features such as CABAC and DirectShow encoding. It is released under the terms of the GPL, but this license is incompatible with the MPEG-LA patent licenses in jurisdictions that recognize software patents.
* * * * *
Curiosity killed the cat, but for a while I was a suspect. —Stephen Wright