Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!


Forgot your password?

ExtremeTech Wages War of the Codecs 356

prostoalex writes "ExtremeTech tested Windows Media, DivX, QuickTime/Sorenson and QuickTime/MPEG4 codecs. They encoded clips from Matrix Reloaded, Monsters, Inc., X2 and Spider-Man. QuickTime/Sorenson won the encoding speed contest, for the quality tests read the entire review, as each movie sample was encoded with 500KB and 1MB bitrates. Video samples provided on the site as well, so see for yourself."
This discussion has been archived. No new comments can be posted.

ExtremeTech Wages War of the Codecs

Comments Filter:
  • But no Xvid? (Score:5, Interesting)

    by Anonymous Coward on Thursday March 11, 2004 @04:38PM (#8535781)
    They expect to have their opinion valued on Slashdot when they don't review the open source video codec? (It generally wins in other tests.)
    • Dont forget ffmpeg (Score:2, Informative)

      by Anonymous Coward
      Dont want to piss off the BSD crowd either!
    • Doom9's Comparison (Score:5, Informative)

      by Anonymous Coward on Thursday March 11, 2004 @04:44PM (#8535866)
      Here's their most recent codec shootout [doom9.org] with 3ivx, Divx, ffvfw, Nero, Real, On2 and Xvid. Xvid wins [doom9.org].
      • what about ogg? (Score:3, Informative)

        by TubeSteak ( 669689 )
        Xvid and 3ivx get mentioned many posts further down, but no one says anything about ogg!!! From my limited experience, it has amazing quality and the greatest advantage over xvid and that damnd WMV9 is that it is instantly seekable. No lag while xvid thinks and no buffering by windows. I personally use media player classic for my WMV's & get instant seeking, but his target audience doesn't.

        All hail ogg!
        • Re:what about ogg? (Score:3, Informative)

          by imroy ( 755 )
          Ogg WHAT?
          Are you talking about using Ogg Vorbis as the audio codec? Yes, it is very good. I wouldn't use anything else for either my CD rips or DVD rips.
          Or are you talking about the bastardisation of the Ogg container format that is the OGM container format? Do some googling. From the mailing list postings I saw, the Ogg guys aren't too happy about this effort by one windows programmer to hack the Avi/VfW information into the Ogg container format. If that's what you're referring to, and using, I recommen
    • Re:But no Xvid? (Score:4, Insightful)

      by rbegga ( 662104 ) on Thursday March 11, 2004 @05:34PM (#8536504) Homepage
      No kidding. Taking into account that they bash the Apple/Quicktime MPEG-4 quality, and that the article is published by Ziff-Davis (Who I consider to be the least credible source for information in the industry), you might as well be reading an article published by Microsoft about how great WMV is.
      • Re:But no Xvid? (Score:5, Insightful)

        by contradyction ( 672874 ) on Thursday March 11, 2004 @09:15PM (#8538445)
        Taking into account that they bash the Apple/Quicktime MPEG-4 quality, and that the article is published by Ziff-Davis (Who I consider to be the least credible source for information in the industry), you might as well be reading an article published by Microsoft about how great WMV is.

        OH NO!!! The article didn't say that Apple is 100% awesome and they didn't replace the 's' in Microsoft with a dollar sign. That must mean that the article was written by Bill 'Son of Satan' Gates himself!!! Or it could just mean that the videos generated by the Quicktime encoder didn't look as good as the ones generated by the WMV9 and the DivX encoder. Look [extremetech.com] at [extremetech.com] the [extremetech.com] comparisons [extremetech.com] for yourself and realise that in this case the Apple tool was a distant third.
    • Re:But no Xvid? (Score:3, Informative)

      by Anonymous Coward
      Maybe because Xvid does not have licences for MPEG4 patents, and therefore isn't really legal in the US.
    • Re:But no Xvid? (Score:4, Interesting)

      by dr.badass ( 25287 ) on Thursday March 11, 2004 @10:12PM (#8538873) Homepage
      Up until very recently, XviD.org, "The Home Of The XviD Codec" didn't have any public binaries, or any links to any. I believe the official stance was "It's not done yet, so nobody should be using it. Piss off."

      Maybe that's why they didn't review it.
  • I smell... (Score:3, Funny)

    by xSquaredAdmin ( 725927 ) on Thursday March 11, 2004 @04:38PM (#8535785)
    a/.ing "video samples provided on the site as well, so see for yourself."
  • ... pale in comparison to ASCII-mation.

    Episode four in under a meg!
  • Hahahahaha (Score:3, Funny)

    by Erwos ( 553607 ) on Thursday March 11, 2004 @04:39PM (#8535800)
    "Video samples provided on the site as well, so see for yourself."

    Yeah, as if there was any chance of THAT happening after you submitted that site to /. Good one!

  • Bah....... (Score:4, Funny)

    by Kenja ( 541830 ) on Thursday March 11, 2004 @04:40PM (#8535808)
    Bah, they should have used the nipple scene from Spider Man. The rest of the movie was a total wash.
  • by garcia ( 6573 ) * on Thursday March 11, 2004 @04:40PM (#8535814)
    I took a look at there sample images and found very little difference (other than MPEG4 obviously) in their quality.

    It is very difficult for ME to decide between them. I have never actually seen any QT movies up for download as far as real movies go. Most movies are encoded with divx and seem to work just fine.

    Do people really care about minor differences in quality when the file sizes are down to 710mb? I know I don't. Blurred motion is just something I deal with when I download something.

    Encoding time is important only if you do this regularly. For those of us just watching a movie it doesn't matter. Whatever gives me the smallest file size with a decent picture is what I want to go w/.
    • by Erratio ( 570164 ) on Thursday March 11, 2004 @04:46PM (#8535891)
      One big factor which is neglected in this article is compatibility. I don't do too much with video files but a decent amount with audio and sometimes there are formats with minor quality differences, but what ultimately decides which to use is how many players can handle it easily. It doesn't come into consideration quite as much for personal archiving and controlled environments, but even then you can't tell what the future will bring and a little flexibilty now saves time later.
      • by badasscat ( 563442 ) <basscadet75NO@SPAMyahoo.com> on Thursday March 11, 2004 @06:18PM (#8537023)
        One big factor which is neglected in this article is compatibility.

        I don't think this was neglected at all; it clearly influenced the choice in what codecs to test in the first place. The four codecs in this article are the four most popular - wmv comes standard on PC's, QuickTime is Apple's standard and works across all major platforms, Divx is still the non-Linux geek's codec of choice (and it works with WMP) and MPEG-4 is now supported in the latest QuickTime.

        The codecs chosen for review, then, are the ones that work with the players used by the greatest number of people. A lot of earlier posters complained about this or that codec not being included here, but they obviously missed this pretty critical point of the article. It doesn't matter to me, as someone working for a commercial enterprise that has to encode videos for our customers, whether Xvid or whatever offers slightly fewer artifacts. Because the fact is my customers probably don't have that codec and aren't going to bother downloading it just for me. Even Divx is probably barely at the saturation point where it's worth covering in an article like this, but for certain purposes and for a certain audience (PC gamers, for example), it's worth considering.

        As others have pointed out, there are articles out there dealing with the lesser-used codecs if you just want to know who the absolute quality winner is. But in the real world and unless you're encoding video only for yourself, whatever codec wins in absolute quality is basically irrelevant. What matters is which codec offers the best quality among those in widest general use, and I thought this was a decent article on that basis (though in all honesty simply seeing the examples is probably good enough - I don't need an explanation of how blocky MPEG-4 is in an image, I can see it myself).

        And it seems to me that what this article is saying is that if you want to use a cross-platform codec that everybody probably has (even on Linux), use plain old QuickTime. If you want to encode for the geek crowd, use Divx. If you want the best quality overall and you don't care about excluding a small percentage of the audience, use WM9. Whatever you do, avoid MPEG-4. Simple, and helpful to any professional whose job includes either encoding or contracting out encoding of videos for customers.
        • by S.Lemmon ( 147743 ) on Thursday March 11, 2004 @07:30PM (#8537684) Homepage
          One thing the article and most people here seem to miss is DivX *IS* MPEG4. XviD is as well - that's why a MPEG4 decoder like ffdshow can play them both.

          The article can really give people the wrong idea - it's not the MPEG4 codec, but maybe Apple's implementation that's to blame. Perhaps it just doesn't support all of MPEG4's features. Then again, perhaps the people doing the review just didn't know how to set up the encoder properly. Regardless of codec, there's quite an art to good encoding.
          • Problem is... the encoding/compression standard used is only one part of the story.

            The actual encoder that is used and the parameters used for encoding are of at least as much importance.

            Then, there is a whole lot of compromises to be made. I am not too familiar with sorenson and wmv9 but for mpeg video you have a lot of things you can tune on the encoder side (for all mpeg versions, tho the actual tunables differ). At any given resolution and average bitrate, you still have a choice to use more I frames,
        • Quicktime != codec (Score:4, Informative)

          by Ohreally_factor ( 593551 ) on Thursday March 11, 2004 @08:58PM (#8538311) Journal
          Just to clarify, Quicktime is a media architecture, not a file format or a codec.

          This misunderstanding doesn't invalidate your argument, although I would disagree with you about MPEG-4. I've gotten good results with it, sometimes even great results.
    • by poptones ( 653660 ) on Thursday March 11, 2004 @05:04PM (#8536129) Journal
      Not all of us use this stuff for DOWNLOADING MOVIES. I haven't downloaded a movie in ages - I don't have the bandwidth. However I have several on my hdd and use xvid not only for archiving music videos and shows I enjoy, but also to get around the general DVD suckiness (movies that degrade over time, stutter, and require me to go back to the disc every time I want to watch one).

      I like having all my movies and music and shows just a mouse click away. No fondling media, no DVD drives whooshing and movies stuttering halfway throgh because some tiny piece of schmutz got on the precious disc. In order to do this, I don't care at all what 500kbs or 1mbps files look like - The Twins effect [lovehkfilm.com] alone occupies about 2GB on one of my drives, and I still haven't been able to produce a rip of Natural City [hkflix.com] that satisfies me even when the last one I tried was nearly 4GB (lots of film grain in that one and I don't care to lose it).

      Yes... many of us care about quality. In fact, this is the very reason I rip DVDs - so the programs I enjoy play (more smoothly) from my hard drive.

    • "Encoding time is important only if you do this regularly. "

      It's also important if you wish to capture video and encode it in real time, a la software based TiVo.

      Don't underestimate that aspect of encoding.
  • I don't care. (Score:4, Insightful)

    by MooKore 2004 ( 737557 ) on Thursday March 11, 2004 @04:41PM (#8535817) Homepage Journal
    Thanks to the hard work of the Mplayer team, I can play any video format I want. If you havent tried it, you should.

    Get mplayer [mplayerhq.hu]
  • by Anonymous Coward on Thursday March 11, 2004 @04:41PM (#8535818)
    How do they expect me to keep pirating Hollywood movies if they keep changing the damn codec?!

  • No XVid? (Score:4, Informative)

    by Rexz ( 724700 ) on Thursday March 11, 2004 @04:41PM (#8535823)
    I'm suprised XviD, an open source, MPEG-4 compliant codec wasn't tested. It's quickly becoming a standard for the transfer of large movies, and its open source nature has all of the usual benefits: alternatives, power and no constraints or adware. I suggest anyone planning on encoding video seriously considers it. XviD.org [xvid.org]
    • by Sycraft-fu ( 314770 ) on Thursday March 11, 2004 @05:15PM (#8536252)
      I mean MPEG-4, despite being open, is NOT a free format. You are required to pay licenses for encoders and decoders. Has XviD payed this? If not, it's technically not legal. That could keep it out of being a serious contender for pro use. I mean I'm betting the MPEG-4 group isn't going to care if some hobbiests are using an unlicensed encoder, but they'll care if pros are.
    • Re:No XVid? (Score:5, Funny)

      by Trogre ( 513942 ) on Thursday March 11, 2004 @05:25PM (#8536385) Homepage
      Hey, I just noticed, that's like diVX spelled backwards!

      Now that's clever.

      Or does it stand for something like:
      X Vid Isn't Divx

  • Made on a Mac? (Score:3, Interesting)

    by Rick Zeman ( 15628 ) on Thursday March 11, 2004 @04:42PM (#8535839)
    I wonder how much better the QT/Sorenson and QT/MPEG4 (and maybe divX, dunno if there's an encoder) testing would have been if they were done on a Mac and the Velocity Engine could have been utilized?
    • Re:Made on a Mac? (Score:3, Insightful)

      by swb ( 14022 )
      How does the velocity engine make *better* encodings using the same codecs as x86? Presuming that the codecs are implemented the same, wouldn't it just maybe do it faster?
    • Re:Made on a Mac? (Score:4, Informative)

      by SlamMan ( 221834 ) <squigit AT gmail DOT com> on Thursday March 11, 2004 @05:00PM (#8536091)
      The reason its poor is that they're using the free version of Sorenson 3, as opposed to the pro version that everybody else in the world doing pro video with shelled out $300 for (and is well worth it).
      • From the article:

        There are simply too many video codecs out there for us to test them all -- and most of them wouldn't be useful anyway. We focused on four codecs, all of which are free and can be used with free tools. (Or very cheap ones - QuickTime requires a $30 Pro registration for full encoding capabilities.) You don't want to pay $500 for a professional video authoring program just to send grandma a video of baby's first steps, so we stuck with these four very popular codecs...

        The article is te
  • by jamshid42 ( 218149 ) on Thursday March 11, 2004 @04:44PM (#8535856) Homepage
    OK, the article blurb claims that QT/Sorenson had the fastest encoding times, but also had the third-worst quality (only QT/MPEG-4 was worse). DivX seems to have the best quality, which, in my opinion, should be the end goal.

    Think about it, how many times are you going to encode a movie? How many times are you going to watch it? Typically, you are going to encode once and probably watch it multiple times. Therefore, I would happily accept a little longer processing time in the beginning if that means I will end up with a better quality production.

  • doom9.net (Score:2, Informative)

    by silverfuck ( 743326 )
    A useful site for all things high(ish) quality video encoding, aimed at dvd backups to cd, is Doom9 [doom9.net] - see their last round of codec comparisons [doom9.org]. (Frame based, so you'll need to click through from the beginning to get the menu frames etc.)
  • by markv242 ( 622209 ) on Thursday March 11, 2004 @04:45PM (#8535874)
    Posting still images isn't the best way to point out video artifacts due to compression. Post five seconds of compressed material (all of this qualifies under fair use) and let the users see the artifacts themselves. The human eye is much more likely to spot the artifacts in a movie because of our perception of motion.
    • by pla ( 258480 ) on Thursday March 11, 2004 @05:25PM (#8536383) Journal
      Posting still images isn't the best way to point out video artifacts due to compression.

      If I considered that as their worst methodological flaw, I'd tend to agree with you completely.

      However, for those who haven't bothered to read the article, two points (well, more than two, but as examples...) stuck out as completely invalidating their results:
      "In fact, even the original high bitrate MPEG2 on the DVD struggles in places."
      "We therefore took the uncompressed clips and created new "master clips" by encoding them to very high bitrate (around 8 megabit) files using Indeo 5.1 compression, as all our test applications could easily read this format"

      Anyone else spot the problem, here?

      First of all, starting with lossy source material automatically injects artifacts into the video. A codec that looks for similar ways to trim bits as the original (MPEG2), ie, MPEG4, will natually have a distinct advantage in having fewer artifacts in the final result. Not that I can think of any means by which they could have obtained high-quality "raw" video, but any valid test of an encoder's capabilities would require them to do so.

      And, as if that didn't introduce enough bias, they then reencoded in Indeo 5 format, before using each "real" codec under consideration. Again, that injects its own artifacts, and favors codecs that look for similar ways to trim bits. But, all four of the codecs tested can deal with MPEG2 as source material, so even the "to make it fair" argument falls flat here.

      Overall, this so-called "comparison" has zero external validity, in the strict experimental sense. They managed to waste a few hours of CPU time, nothing more.

      At the very least (if they couldn't get ahold of raw HQ video), going straight from MPEG2 would have given a meaningful comparison of "how it will look ripped from a DVD". But by the Indeo pass, they removed even that as a possible claim.
      • by Refrag ( 145266 ) on Thursday March 11, 2004 @09:10PM (#8538411) Homepage
        You're right. They should have started out with either from a DV camera or the MPEG2 from a DVD. There was no reason to go to some other stupid codec before re-encoding the material a third time.
  • by SideshowBob ( 82333 ) on Thursday March 11, 2004 @04:45PM (#8535876)
    Sorenson Pro (which has 2-pass and VBR encoding) isn't available in the $30 QTPro package. Use Sorenson Squeeze or MediaCleaner.

    Also, QuickTime's MPEG-4 encoder is not the best MPEG-4 encoder out there. But there are better ones available, and of course MPEG-4 being a standard, the output of those other tools will be playable in QT Player.

    So to make the comparison valid, both in terms of encoding speed an quality, some other tool should've been used.
  • Thanks, I think? (Score:4, Insightful)

    by MalaclypseTheYounger ( 726934 ) on Thursday March 11, 2004 @04:45PM (#8535877) Journal
    What a wishy-washy article. To sum up and save you the 2 minutes of your life to read that article, all 4 techs are good, and they are all good for something, bad at others. I wonder if the author could have sat on the fence any MORE when comparing the codecs.

    I for one, will continue to obey my DivX Masters, they have always been good to me. It seems that the author had a hard-on for QuickTime and M$, both of which annoy the ever living crap out of me... QuickTime, with it's little icon in the toolbar that just won't go away, and Microsoft because I just can't trust them to not tell MPAA which movies I have on my HDD.

    $0.02 Flamebaiting, Trolling response concluded.
    (and my Karma just got back to Neutral, dang it)
    • My problem is encoding at 500KB or 1000KB, that makes for a large as movie. I encode a quality based Xvid and that usually comes out pretty well, what codecs really need to master is the art of using high KB when it's needed and low KB when it's not, also, would you people stop encoding the damn black lines into the movie. lol. just use apropriate ratio and my player takes care of the lines, that way I don't have 300 MB of encoded black.
  • Come on! (Score:5, Funny)

    by Udo Schmitz ( 738216 ) on Thursday March 11, 2004 @04:46PM (#8535894) Journal
    for the quality tests read the entire review,

    Come on! Are we now suddenly supposed to actually read those frickin' articles? Just tell me who won. This is the internet and my attention span ... wait, what's that shiny thing? ...

  • Xvid rules the scene (Score:4, Interesting)

    by Anonymous Coward on Thursday March 11, 2004 @04:46PM (#8535895)
    You only need to look at the scene to know what codec is in this month. It's Xvid by a long margin, most TV-shows (Stargate, Enterprise, whatever), DVD-rips, Anime etc. are encoded with it. MP3 and AC3 are predictably dominating the sound codec, with many TV-shows now also making the transition to full AC3 (well, the DVD rips of them at least).
  • Stupid test (Score:4, Insightful)

    by stratjakt ( 596332 ) on Thursday March 11, 2004 @04:46PM (#8535896) Journal
    Test of some scenes from copyrighted movies? What're the results for? What format is best downloaded off of Kazaa?

    And the samples are all live action.. Test encoding some hand drawn animation (ie; an old bugs bunny), a computer generated animation, a anime style animation, a dialogue type scene, a live action scene with a lot of action, black and white vs color, etc, etc.

    The types of images on screen greatly affect the performance of different algorithms.

    Plus, each codec has about a million tweaks and optimizations for different types of footage.

    I doubt highly that there's one clear "winner". It's really not that simple.

    Which is why I hate sites like ExtremeTech that always have to boil it down to "this product is the best, the rest suck!".

    Like the ATI vs nVidia flamewars. ATI may benchmark faster, yet nVidia has effects in games ATI lacks. There is no clear "this one is the best". Or Intel vs AMD or Linux vs Windows, etc, etc..

    Nothing in the realm of computer science is that simple.
    • Re:Stupid test (Score:3, Insightful)

      by dasmegabyte ( 267018 )
      Oh please. If they didn't pick a winner, fanboys wouldn't have anything to pout about.

      Personally, I gave up gloating about my superior hardware when 3dFX fell apart. I looked at my very nice 16 meg TNT2 graphics card and said to myself, "You know? Without the Voodoo2, this card would never have been made. The TNT1 was sped up to take market share from the Voodoo1. The GPU was invented to stave off ATI and PowerVR. Competition resulted in me getting a better product faster."

      So you know what? I don't
    • by Trogre ( 513942 )
      And the samples are all live action..

      What, you mean Mike Wasowski is real? I knew it!

  • by ebrandsberg ( 75344 ) on Thursday March 11, 2004 @04:47PM (#8535920)
    Unfortunately, due to the fact that the source was itself the result of mpeg encoding, it could unfairly impact the ability of the various codecs to handle the content. On the flip side, much of the content people are encoding is actually decoded content, i.e. from a digital camcorder, etc.

    What would be interesting is taking the original raw film footage (that hasn't been digitally compressed with a lossy method) and encoding, then comparing the results.
    • This is exactly what I was going to point out. The test is actually showing which codec is the best at recompressing MPEG2 (which is still a relevant test when you are talking about moving a DVD onto your handheld or something), not which is the best for compressing raw uncompressed footage. In recompression, if you use two encoding algorighms that use much different techniques, often you lose a lot of the detail of the original. Use the analogy of rotating one polarized lens over another... Both lenses rem
  • On2 VP4 (Score:4, Insightful)

    by Anonymous Coward on Thursday March 11, 2004 @04:49PM (#8535939)
    As sad as it sounds, I'm not surprised that XviD was left out. After all, alot of these reviewers pay attentions to what is being *marketted*. But I am surprised that On2 [on2.com] failed to get their VP4 Personal Codec [on2.com] noticed by this reviewer. I guess On2's marketting group dropped the ball when it came to make ExtremeTech aware of it's offerings.
    • ExtremeTech is not your truly commercial site. A lot of their stuff is written by volunteers who put in time and effort to get published. I am not exactly sure about this author, he might work for PC Mag, which owns ExtremeTech, but generally the site is quite unbiased.
    • And VP6! (Score:3, Informative)

      by benwaggoner ( 513209 )
      VP4? I don't believe that was ever released. I had a review copy of it, but they quickly superseded it with VP5.

      VP3 was the one that was open-souces, and is used as the basis of Ogg Theora.

      The current On2 codec is VP6, which is free for personal use.
  • What really matters. (Score:4, Interesting)

    by FreeLinux ( 555387 ) on Thursday March 11, 2004 @04:50PM (#8535957)
    Ok. encoding speed is somewhat important to a few people and bitrate is pretty important to most people but, quality is the most important to almost everyone. From a quality standpoint DivX is the clear winner. But, it still isn't broadcast quality let alone DVD quality.

  • by NaugaHunter ( 639364 ) on Thursday March 11, 2004 @04:52PM (#8535978)
    Aren't there any 'Stock-footage' type DVD's out there that include a license to redistribute they could have used? Or couldn't they at least have tried to get permission for the clips they used?

    While they might not have want to try to argue fair use through education or reviewing, they could have found at least one clip they could distribute. Hell, rent a high end digital camera and make one. Tape traffic on a highway, both daytime and nighttime, and you've got a motion video test, or a fountain, or anything.
  • No RealPlayer?
  • Though we have certainly examined these four video codecs extensively, this article can by no means be considered the be-all, end-all on their relative merits. There are dozens of parameters and options when it comes to encoding video and, as thorough as we have been, there is much left unexplored. Variable bitrate and dual-pass encoding, for example, are common and useful features we didn't cover here - unfortunately, they're not entirely available in QuickTime 6.5 Pro for Sorenson3 or MPEG-4 encoding.


  • Give it some time (Score:4, Interesting)

    by diamondsw ( 685967 ) on Thursday March 11, 2004 @04:53PM (#8536006)
    Count me in as surprised at how poorly Apple's MPEG-4 implementation did. However, as a very new codec I expect it will improve in time. Or Apple will simply license someone else's codec.

    Regardless, Apple has been one of the biggest supporters of MPEG-4, and I thank them for that.
    • Apple's MPEG-4 decoder is very good and fast, for Simple Visual profile. Their encoder is what's lacking, being 1-pass only, and tuned for speed instead of quality.

      Most MPEG-4 professionals would use something like Squeeze or Compression Master instead to make a .mp4. Way better results with identical compatibility.
  • Doom9's Comparison (Score:4, Informative)

    by kylethemile ( 149934 ) on Thursday March 11, 2004 @04:58PM (#8536053)
    Well, seeing how bad [hydrogenaudio.org] ET's iTunes Bad, WMA Good [extremetech.com] article was, I figure Doom9's codec comparison [doom9.org] is better than this.

    And yes, Doom9's comparison includes XViD.
  • Sorry (Score:5, Insightful)

    by pclminion ( 145572 ) on Thursday March 11, 2004 @04:58PM (#8536061)
    Sorry, but any "test" where there's no scientific definition of what an "artifact" is, nor any mathematical definition of "image quality," is total bull. Yes, it's important to include subjective experience in the criteria, but we also need hard numbers. Where are the hard numbers on luminance distortion? Chrominance distortion? How many bits per pixel do you pay for each decibel of noise reduction? What's the worst case performance (no correlation between frames)? Best case performance (no difference between frames)?

    I'm sorry, but some hand-waving, subjective "Hey, this thing kinda looks better than that thing" is not a test. Calling it a "War of the Codecs" is even more ridiculous.

  • by David_Bloom ( 578245 ) <slashdot@3lesson.org> on Thursday March 11, 2004 @05:05PM (#8536143) Homepage
    We therefore took the uncompressed clips and created new "master clips" by encoding them to very high bitrate (around 8 megabit) files using Indeo 5.1 compression, as all our test applications could easily read this format.
    Indeo? INDEO!??!? Yes, I know if you make every frame a keyframe or whatever, maybe it would look almost decent. But seriously - why not use a JPEG series or something instead? I'm sure both QuickTime and VDub can handle that. In fact, if you had bothered to discover VirtaulDubMod [sourceforge.net] and the QuickTime MPEG-2 playback component [apple.com], you could have just plugged in the MPEG-2 streams directly.


  • I call shenanigans (Score:5, Interesting)

    by awaspaas ( 663879 ) on Thursday March 11, 2004 @05:07PM (#8536167)
    Okay, I just encoded some DVD-size video at 1mbit and 500kbit, 1/4 size, in QuickTime MPEG-4 and can barely see any artifacts in either. This dude seriously got some settings wrong in his MPEG-4 encodings, although I don't quite see how that's possible as settings are quality, framerate, keyframes, and data rate (and he said quality was set at best). I'll post some screenies later if I get a chance.
  • by Anonymous Coward on Thursday March 11, 2004 @05:11PM (#8536210)
    The article's not very great. First, they convert the MPEG-2 stream to INDEO, then from INDEO to whatever the target is. Fine, but the process isn't a transitive one. Some codecs will not produce as good an input for other codecs, thus biasing the results.

    What's particularly suspicious is that Apple's MPEG-4 came out so poorly, though WMV9, and DivX are nothing more than early MPEG-4 codecs. Sorensen3 is the only substantially different algorithm used. And why use MPEG-4? It was originally designed for low-resolution low-bitrate applications (PDAs, cell-phones, etc.)

    Why so slow? I do most of my video transcoding under Linux, but they aren't getting much better throughput than I do, and their machine's at least 4 times as fast as mine? I suppose it's got to do with using Indeo (my source is DV), so there's an extra decode step, but it's still quite slow.

    I've distributed a number of my own videos in the MPEG-4 format, and don't see the sort of horrible results they demonstrated in their examples -- but then again, perhaps I do preprocessing (quantization, denoising, etc.) that they don't include in their process.

    Regardless, my personal experience is that at high or low bitrates, most of the codecs are interchangable. Perhaps you need to fiddle with the encoding parameters, but you can almost always get results close enough to identical as not to matter. It becomes more difficult with mid-range bitrates (2-3Mbps@720x480x29.97) that some codecs show strengths over the others. In that department, I almost always go with MPEG-2 with custom quantization matrices...
    • Taxonomy of MPEG-4 (Score:3, Informative)

      by benwaggoner ( 513209 )
      WMV9 isn't MPEG-4 derived. MS MPEG-4v3 was, but that forked into WMV7 years ago. WMV9 is quite different than MPEG-4 now.

      QuickTime encodes and decodes Simple Profile MPEG-4

      DivX did Simple in V4, and V5 added support for Advanced Simple.

      Most of this will be moot soon, since the MPEG-4 Part 10/AVC/H.264 codec is way better than the old Simple or Advanced Simple, and will rapidly replace the old versions in the next couple of years.
  • Limited value (Score:5, Interesting)

    by Dr.Knackerator ( 755466 ) on Thursday March 11, 2004 @05:15PM (#8536253) Journal
    Does it tell you which codec is best? Maybe but only for recompressing MPEG-2 footage. They *should* have tested against DV output as the standard consumer format, and uncompressed video. Plus looking at snapshots of compressed movies is of limited value, there is a big difference between what detail we can determine through a still image and a moving one. If you were to freeze a tv picture (or look at a captured frame) which includes something moving you would see a combing effect of the interlaced video. It doesn't look like that when you view it though.
  • by sjonke ( 457707 ) * on Thursday March 11, 2004 @05:28PM (#8536421) Journal
    I have tried using DiVX and Apple's MPEG4 to encode a short video and there are just far too many options to play with. It's virtually impossible for the average person to use any of these and get great results. We need something that will produce excellent results at the click of a button. Until then I'll stick to showing my digital videos saved back to the camera, plugged into a TV, where quality is fantastic.
  • by potmos ( 604320 ) on Thursday March 11, 2004 @05:35PM (#8536510) Homepage
    I didn't read the article, but after looking at the chart, it is clear that Monsters, Inc. clearly beat out those other movies. It's over 4x faster than Spiderman.
  • by SensitiveMale ( 155605 ) on Thursday March 11, 2004 @05:35PM (#8536514)
    in spiderman.

    Kirsten Dunst's wet t-shirt scene.
  • by gsfprez ( 27403 ) on Thursday March 11, 2004 @08:04PM (#8537950)
    I know this was strictly about quality - but i think its very shortsighted to miss some of the key features WHY people like Quicktime... and its not always about the quality of watching ripped DVDs.

    For example - I don't see anywhere where it points out that Quicktime and Divx are by far the most DRM-less codecs out there. WMP9 can stick you up the ass if you're not careful. Plus, there are plenty of times that WMP9 will refuse to operate properly with multi=monitor setups (my friend's brand new ThinkPad, for example, refuses to play over the external VGA port....)

    i also don't see any mention of the ability to cut/copy/paste with the built-in default players as a comparison tool. How many times have you wanted just a sliver of a movie to playback - ro to have the ability to quickly convert it to DV to put onto a workflow with some other editing? Even the average goofball making iMovies wants to do that all the time - but is 100% prevented from doing that with WMP9

    While the quality of QT is obviously lacking - i'll use it EVERY DAY OF THE WEEK because its far more powerful for everyday use, and much more free of DRM issues.

    this would explain, of course - Hollywood's facination with it - its got great quality while sacrificing little things... like fair use.
    • Dude, the article is about codec comaprison, not player comparison. And you don't need WMP9 (Windows Media Player 9) to play WM9 (Windows Media V9) encoded files. Heck, you can even encdoe videos with WM9 into .avi or .ogm or whatever open source free beer speech wrapper format you want by using the VCM [microsoft.com] wrapper released by Microsoft.

      The codec itself is neutral from any copy protection mechanism, or you just like to yell "DRM" for some cheap mod points.

  • What the? (Score:3, Funny)

    by (eternal_software) ( 233207 ) on Thursday March 11, 2004 @09:22PM (#8538491)
    From the QuickTime summary:

    Pros: decent image quality
    Cons: Horrible image quality
  • by benwaggoner ( 513209 ) <ben DOT waggoner AT microsoft DOT com> on Friday March 12, 2004 @12:39AM (#8539896) Homepage
    Okay, so I AM the world's leading expert on video compression codecs and formats (no, really, I am). I cover the same ground in my book, and in a series of articles for DV magazine over the last five years. So I'm pretty picky on this kind of things. But these guys couldn't compress themselves out of a wet paper bag.

    Some fundamental errors:

    They're using MPEG-2 sources, which are already highly compressed (this has been amply covered by other posters).

    They talk about converting to an "uncompressed" AVI, but never specify which flavor of uncompressed. They should have used a lossless codec that uses the native Y'CbCr color space of video, like Huffyuv. They way they just said "uncompressed" suggests they used the AVI "None" codec, which is uncompressed RGB. This causes two lossly color space conversions - one from the Y'CbCr of the source to RGB, and then back to Y'CbCr in the delivery codec.

    They used Indeo 5.1 as their intermediate codec. This is terrible. Indeo uses what's called YUV-9 sampling. There is only one measurement of color per 4x4 block of pixels. This throws away 75% of the color information from the DVD (which uses 4:2:0 sampling, with 2x2 blocks), before it even touches a codec. And this results in very ugly blocks whenever there are highly saturated regions with sharp contrast. So, all the output is going to look highly compressed when rendered from these intermediates, even if further compression is lossless. Look at the Spider Man test frame for an example. Notice the red blooming around the shoulders of the vocalist. And the color everywhere is very muddled. Indeo can also be slow to decode, unless it was encoded with all keyframes. And how slow it is to decode will vary with the tool, which probably added measurable error to their encoding time measurements.

    They don't know the difference between Sorenson Video 3, which comes free with QuickTime, and Sorenson Video 3.3 Professional, which you have to pay for and is what Apple uses for their movie trailers. With the Pro version, critical features like B-frames and 2-pass VBR are available.

    Apple's MPEG-4 encoder isn't very good - 1-pass only, tuned for speed more than quality. A file with the exact same compatibility could be made with Squeeze, Compression Master, Envivio, etcetera with MUCH better quality. And the Divx MPEG-4 codec is, of course, also MPEG-4.

    They didn't use 2-pass encoding! No quality-concious encoder would ever put content on spinning disc without using 2-pass. And they didn't mention most of the other encoding settings they used, which by context I'd guess were basic defaults.

    That's from an initial skim. If I spent more time with the article.

    In summary, these guys spent hours and hours analyzing the results of tests, where they would have been WAY better off spending an hour asking someone who knew anything about video compression how to administer this kind of test.

    Oddly enough, their results are vaguely like you'd expect - WMV9 and DivX do well, Sorenson less so, and Apple MPEG-4 at the rear. Done properly, I imagine WMV9 would have had a slight lead, and Sorenson 3 Pro would have been a lot closer to DivX. And no one uses Apple's MPEG-4 codec for content distribution. QuickTime's decoder is fine, so folks would use a professional-grade MPEG-4 encoder instead.

"Call immediately. Time is running out. We both need to do something monstrous before we die." -- Message from Ralph Steadman to Hunter Thompson