The average mp4 vs a highly compressed digital cable channel might be equivalent, but the market wants DVD quality without any sacrifice from downloaded video.
As far as the audio situation being different from video, one of the reasons is that audio has been distributed (very near) DRM free and damn good quality ever since cds. So there was made an infrastructure for people putting up pirated versions ripped from the high quality originals.
Video is different. DVD is mpeg2, which isn't all that great, but much of the pirated video out there is reencoded with this as the original source. So to keep a decent amount of quality, you have to use larger file size, even with these superior codecs, because the original isn't exactly spectacular in terms of quality/size. And the industry refuses to put stuff out in anything better than mpeg2 without making it a royal PITA to use. So we're not taking full advantage of new codecs and getting the mileage out of them that we should be. This is changing, but takes time.