kidjan's Journal: Why video conferencing isn't going to happen any time soon.

Before I'm savegely beaten for the title alone, let me first say a bit about what I do for a living.

I currently work for a company that is designing and implementing an IP-based security system. As such, we stream video over an IP based network using an RTP library that I adopted and implemented. I have extensive experience with video compression technology and streaming media over IP networks--especially lossy, erratic networks--from a programmatic perspective.

That being said, there are some fundamental roadblocks to having video conferencing. The common retort to this claim is "Well, we currently have audio--what's so different about audio and video transmission over the Internet?" Quite a bit, actually--we'll go into that a bit more.

The problem involves several factors:
1. A lack of bandwidth on people's upload connections to the internet. While many ISPs provide people with three and four megabit connections to the Internet, often times their upload rate is no more than 256 kb (kb being kilobit, KB being kilobyte). While audio is perfectly happy residing in 30-50 kb, video (in any acceptable quality) is not.

2. The Internet itself is unsuitable for delivering large video in a timely manner. The Internet is a big, lossy, trashy, inconsistent network. Packets arrive out of order. Packets are lost. Data arrives at inconsistent intervals. TCP does a lot to correct this, but while TCP has respect for guaranteed delivery, it has very little respect for timeliness of delivery. For a real-time application, like video conferencing, this is bad. If retransmitting lost data ends up taking 200 ms, on top of the rest of the time, the quality is degraded.

3. Lastly, while there have been huge advances in video compression technology, there are tradeoffs for everything. An MPEG4 stream is going to look better than a MJPEG stream at half the bandwidth, but the MPEG4 stream has qualities that make it significantly worse in several aspects. Codecs have what we call "loss tolerance," or their ability (or lack thereof) to loose data mid-stream and still be able to render. Because MJPEG is just a series of frames with no relationship to previous or future frames, loosing a frame isn't a big deal. In fact, it's not a problem at all. You skip the frame, and move on. With MPEG4, however, you have frames that rely on previous frames--loosing one isn't an option.

Combine #2 with #3 and you have a big, big problem. How do you reconcile a trashy, lossy network that doesn't deliver stuff on time with a codec that's completely intolerant to loss? Furthermore, even an MPEG4 stream will need 300+ kb/sec to look decent (we're talking 320*240--nothing spectacular), so we're already in conflict with #1.

My prediction: video conferencing will not be solved until loss tolerant codecs are developed with low bandwidth (not likely), or the Internet itself has a huge increase in reliability and/or throughput.