AAC vs. OGG vs. MP3

asv108 writes "Yesterday, Apple unveiled their new music service claiming that the AAC format "combines sound quality that rivals CD." Here is a little comparison of lossy music codecs, comparing an Apple ripped AAC file with the commonly used MP3 codec and the increasingly popular OGG codec. Spectrum analysis was used to see which format did the best job of maintaining the shape of the original waveform." Wish they had WMAs in there too. And for the spoilage, it looks like OGG comes out on top.

Hard To Tell Difference

I've got a nice pair of Bose headphones, and I listened to an Apple Store AAC file and an OGG version of the same song. I don't consider myself a real audiophile, but it's damn near impossible to tell the difference between the two; though I can definitely hear the improvement from MP3 to AAC or OGG.

Re:Hard To Tell Difference

You would probably notice the difference if you try the test with certain kinds of classical music...

Even then you would probably have to be selective. Rich orchestral works (say, Janacek, Mahler, Sibelius) won't show an obvious difference, but something more spare (e.g. Debussy string quartet or a good recording of baroque strings) will show a big difference that should be evident even on poorer quality equipment.

Re:Hard To Tell Difference

To do a true test, you need to encode the files, decode them to PCM wav format, then burn to an audio CD.

Then, you have to do a blind test with all of them. You also need to use a variety of source material, because different genres of music compress better under some encoders.

Re:Hard To Tell Difference

To do a true test, you need to encode the files, decode them to PCM wav format, then burn to an audio CD. Then, you have to do a blind test with all of them. You also need to use a variety of source material, because different genres of music compress better under some encoders.

If you have to do all that to tell the difference, doesn't that kinda tell you something?

Re:Hard To Tell Difference

That's there value in ruling out variables when trying to objectively compare things?

Re:Hard To Tell Difference

The problem in your test is that if you know which file you're listening at, you're just not fair in your comparison and by listening several times, your brain just makes you hear stuff that is just not there.

A test was made where people would listen to two WAV file, one supposedely was an MP3 (that was expanded to a WAV). 25% of the people could hear a difference between the two WAV files where they were actually the same...

Re:Hard To Tell Difference

To do a true test, you need to encode the files, decode them to PCM wav format, then burn to an audio CD.

Then, you have to do a blind test with all of them. You also need to use a variety of source material, because different genres of music compress better under some encoders.

Or you could just use ABX [pcavtech.com]. That's actually the de facto standard for comparing audio compression. (See HydrogenAudio [hydrogenaudio.org].)

Re:Hard To Tell Difference

That may be good for you and I, but a true audiophile can tell the difference between a pressed CD and a CD-R... ;-)

Well, I'm off, I have to get some gold cables.

Re:Hard To Tell Difference

Then, you have to do a blind test with all of them. You also need to use a variety of source material, because different genres of music compress better under some encoders.

I don't disagree with you, but I just wanted to throw in my own 2 cents worth of informal experimentation:

I recently discovered the sourceforge cdex ripping software, so I finally had a chance to rip all my music to the superior sounding ogg format instead of mp3. Before doing so, my wife and I ran a couple double blind tests with one another to see where the best encoding was.

The only pair of speakers I had to test this was a pair of old Yamaha YST-M7's. These are Yamaha branded $20 single driver computer speakers that came with some computer I bought a while ago. They are pretty bad speakers. For the test, I selected a reasonable genre swath of music:

Dixie Chicks "There's your Trouble"
Oingo Boingo "On the Outside"
Samuel Barber "Adagio for Strings"
W. A. Mozart "Queen of the Night's Vengeance Aria"
REM "Nightswimming"

Each piece was selected because of particular aspects of song such as use of strings, use of horns, or use of voice. Each song was tried in a variety of encodings in both ogg and mp3, constant and variable bit rate, with the original CD wav file thrown in amongst the samples. The mp3 encoder was Lame v 1.27 engine 3.92 Alpha 1 MMX, the ogg encoder was Ogg Vorbis DLL Encoder v 1.09 enging 1.05.

The results strongly disagreed with conventional wisdom. In every case, across genres, on these low end speakers, 320Kbps mp3's were the only ones that fooled our ears. Low bit rate ogg and mp3 recordings were different, but we didn't take time to notice which was better... they were both unquestionably inferior to the source material. Ogg's 350Kbps encoding was good, but inferior to the smaller 320Kbps mp3 files of the same work.

Reading some of the posts on this article, I am rather shocked how many people find sound reproduction to be anywhere between "very good" and "excellent" on mid end equipment listening to 192Kbps encoded audio.

After running this experiment, I ripped about 30 of my CDs to 320Kbps mp3's and noticed another benefit to CD quality rips: I could listen to the music longer without my ears feeling fatigued. I had always thought that it was pumping sound directly into my head from my headphones that caused my ears to become tired of the music. For whatever reason, it takes much longer now. Perhaps 3 or 4 hours compared to 1 to 1 1/2 before.

Re:Hard To Tell Difference

Well your first problem is your headphones, they are distorting the crap out of any music source, go get some Sennheisers, they start around $60 for a good pair of open cans. Also if you are using anything but Fraunhoffer or better LAME for mp3 its just not fair. Btw, I've found high range problems with OGG that were not present in my Lame mp3's (I did A,B,C blind tests on a variety of samples and found a couple of problems with OGG which I reported with samples). AAC at 128kbit sounds like trash just like every other codec at 128, get around 200kbps VBR or 256 CBR and thats where the differences start to really show up (ok they show up at the very low end like 90kbps too but I don't even want to think about that)

Re:Hard To Tell Difference

I've got a nice pair of Bose headphones, and I listened to an Apple Store AAC file and an OGG version of the same song. I don't consider myself a real audiophile, but it's damn near impossible to tell the difference between the two; though I can definitely hear the improvement from MP3 to AAC or OGG.

One, your headphones suck. Bose sells overpriced junk. People think it is good because it is well marketed. If you compare Bose speakers with equally priced speakers from any quality manufacturer, the difference is amazing.

Bose is a scam, and the fact that they are so popular shows how easy it is to run a massive deception against the American people.

Re:Hard To Tell Difference

I completely agree that Bose isn't all its cracked up to be. However, their mediocre sound quality is a more recent thing -- up until the early 80s, Bose speakers were well-built with excellent sound. I have a set of 601s from 1976 that have the best sound of any speakers I own. They're fantastic, but I wouldn't even consider any other Bose speakers.

I have the Bose headphones as well, and they aren't so bad. They're not $150 good, but they're pretty good for mass-market headphones. They're great for gaming -- comfortable and well-articulated sound, just not audiophile quality for music.

Then again, audiophiles wouldn't be listening to MP3s or AACs or OGGs anyhow :)

Re:Hard To Tell Difference

After spending an hour in the listening room in the Bang & Olufson store in my local mall, I gave up and went to the Bose store. Ten minutes later I walked out with a Lifestyle 25 system.

For the record, the tiny Bose Acoustimass speakers are able to hit both highs and lows that were unreachable with anything in the Bang & Olufson store. People think Bose is good because Bose is good. No, Bose does not produce the best speakers in the world, but neither do they claim to - they claim to provide clear, room-filling sound with a good range. And they do. Oh, and the Bose Tri-Port headphones suck. They're a cheaper (and lower quality) knockoff of Bose's own QuietComfort noise-cancelling headset, which is a great product.

[asbestos underwear]
Don't give me any crap about how the QuietComfort headphones raise the noise floor for listening, either. They are one of the best active noise-cancelling sets on the market, and *no* passive system can beat them. Why? Passive systems can't even *begin* to fight bone conduction. Neither can the Bose, but it can produce limited cancelling frequencies to mute bone conduction. And the headphones sound just *great*. Speaker snobs, flame away...
[/asbestos underwear]

Re:Hard To Tell Difference

After spending an hour in the listening room in the Bang & Olufson store in my local mall, I gave up and went to the Bose store. Ten minutes later I walked out with a Lifestyle 25 system.

Uh... imagine that. You went from possibly the worst, most highly overpriced speaker/electronics line to the second worst... and it was better!

Now go try Paradigm, B&W, PSB, NHT, or any other good but reasonably priced speaker line and you'll see why Bose has such a crappy reputation. Be aware of sound levels too -- the most common trick Bose stores pull is demoing the Bose speakers at one sound level and other speakers at another (lower). The louder system will almost always sound better due to psychology.

Bose isn't inherently shitty... it's just shitty for the price. You can get much better stuff at the same price, or the same quality stuff at about half the price.

To be fair...

Don't forget that Apple's AAC's aren't ripped from 48.8 16-bit AIFF's, but re-mastered directly to AAC.

Ripping from the source a disadvantage? Huh?

Let's see. Given the task of creating a codec de novo and the financial and political means to have access to the original source material rather than a version sent through a horribly non-linear sampling mechanism out of your control and beyond your specification, which would you choose?

I'm sure most Slashdot readers will be familiar with the Nyquist limit and understand the complete inability to represent information above the limit, but how many are familiar with the degradations that occur near the Nyquist limit when you have non-infinite signal lengths? This is why oversampling is so important. In general, if you have a signal at frequency f that you want to accurately capture, you should be sampling (by rule of thumb) at 5f or greater. If you sample at lower frequencies, the distortions in phase and amplitude are difficult to predict and statistically analyze as they tend to have uniform rather than Gaussian distributions.

So again, I re-pose the rhetorical question: given the task of creating a new codec rather than rewriting an old one, wouldn't you want to use the least-filtered signal possible as a source, especially when the extant filtering is non-linear, and be able to select by design which parts to encode and which parts to ignore? I sure would.

Re:To be fair...

CD-DA isn't AIFF. CD-DA contains either 2 or four channels of 16 bit audio, sampled at 44.1 kHz, organized into blocks of 2352 bytes. It's big endian (unlike *.wav).
AIFF is a rather more involved format [swin.edu.au]. One of those formats is 16 bit, 44.1 KHz audio.
The only benefit I could see to encoding directly from masters is that it is possible that the "master" could be less prone to jitter. It is concievable that higher resloution masters would be available (96Khz/24 bit) and the encoding process could take advantage of this extra data somehow.

bleat

Well then that bolsters my original reaction, which is that regardless of the original source of these 'test samples', you'll be hard-pressed to lease the master and rip directly to .ogg or .mp3 like Apple has done with the AAC's available off their service.

That's all very well but

Some decent quality properly blinded listening tests would be more interesting than a graph though.
When VHS established dominance of the video market, there were high barriers to change - your player and media were committed to that format.
There are far less barriers to change in the ripped audio format, although there will still be some inertia, but there is nothing* to stop ogg vorbis becoming the dominant format.

Where's my ogg pod then?

* apart from the silly name.

It's Vorbis, not Ogg.

Ogg is a container format. I could in theory put an ACC audio file into an Ogg container.

The audio format you're babbling about is Vorbis. Usually .ogg because it is inside an Ogg container.

Hell, it's not just a silly name problem, it's an entire naming convention issue.

your ogg pod is here

You can get an ogg pod here [sourceforge.net]. ok, it's a little rough, but it's getting better.

Re:That's all very well but

A few years back, Consumer Reports did an interesting set of listening tests. The usual blinds, of course. But the interesting part was that in addition to random staffers, they had two extra groups: sound engineers and musicians. They reported that these two groups differed radically in their rankings of sound quality. The difference was fairly straightforward: The sound engineers gave a high rank to equipment that produced the sound accurately. The musicians gave a high rank to equipment that made the music clear. These are not at all the same thing. In particular, musicians generally liked "distortions" that removed non-musical information, strengthened the fundamentals, and so on.

From a musician's viewpoint, one of the real frustrations with just about anything published about sound quality is that it's always written from the engineer's viewpoint. But what I want to know is which gadgets do a good job of reproducing the music. They never seem to tell you that.

Re:That's all very well but

If the musicians like it that way, they should record it that way. Sound reproduction equipment should do just that -- reproduce as accurately as possible the sound on the CD (or other source)

If people deliberately want to alter the sound, that should be done by effect processing that can be turned off, but not built in by inherent limitations in the reproduction equipment.

Now, if you are interested in sound production, that is another matter entirely. The sound of a (say) guitar amplifier is as much a part of the musician's instrument as the guitar, though it would still be nice if a lot of that load could be taken off of unreliable power amplifiers and placed on reproducable, removable low level effects processing.

Re:That's all very well but

musicians are legendary for listening on poor equipment and filling in the rest of the music mentally

Yes, of course, but there's a different interpretation of this. It's not unusual for musicians to intentionally use low-quality equipment in order to make the music clearer. They aren't overcoming the limitations of the poor equipment; they are using it as a tool.

As an extreme example, I've known a number of musicians who have recordings of harpsichord music, but don't like the instrument itself. The reason is simple: They have good high-frequency hearing, and a live harpsichord is just a loud high-pitched buzz with barely-audible music in the background. But with a recording, especially on low-quality playback equipment, you can wipe out the high frequencies. This makes the music audible.

There are a fair number of people who have a similar reaction to violins. Although it's not as bad as a harpsichord, a violin has strong high-frequency harmonics that are often badly out of tune. If you clip off everything above 15 kHz or so, you eliminate this distracting noise and the music comes through.

I've made a lot of "very live" recordings of dance bands with a room full of dancers. One of the tricks that I've learned is to use fairly cheap mikes that don't hear the low or high frequencies. Then I don't have to do as much processing to get a good sound.

An interesting thing about this: I've occasionally made two recordings, one with good mikes and one with poor mikes that fall off around 12 or 14 kHz. When I play them for listeners who were there, they inevitably say that the "poor" recording sounds more live than the "good" one. What seems to be going on is that the human brain is fairly good at compensating for the low- and high-frequency noise in such situations. Participants don't hear all the background noise. But in a quiet room with the noise coming from a speaker, people do hear it.

This is similar to the phenomenon that photographers will tell you about: The human eye/brain system is very good at correcting for color cast. Cameras record the true color (within the bounds of the film type and latitude), so the cast is visible in the photo when it wasn't in the original scene. But photographers learn to see the full color and can't ignore a color cast, just a musicians learn to hear all the sound and can't easily ignore background noise.

(Similarly, after playing around with a polarizing filter for a few months, I found that I could "see" polarization. And now I can't turn it off. ;-)

It's all very complicted.

Re:That's all very well but

This is similar to the phenomenon that photographers will tell you about: The human eye/brain system is very good at correcting for color cast. Cameras record the true color (within the bounds of the film type and latitude), so the cast is visible in the photo when it wasn't in the original scene. But photographers learn to see the full color and can't ignore a color cast, just a musicians learn to hear all the sound and can't easily ignore background noise

Well this photographer will tell you differently.

If you use film stock then a very important part of the printing process is setting the filters to give the correct colour balance - either by hand or by bulk scanning the film and normalizing to 18% grey.

On a digital camera or video camera you have to set the white balance so the camera electronics know the reference to record the colour signal against.

Neither film nor CCDs/CMOS sensors have anywhere near the dynamic range of the human eye, so they record a substantially less accurate picture with either the highlights or shadows saturated out.

The only way of accurately scientifically measuring the scene is with a multispectral scanning radiometer - as used in remote sensing.

Speaking as a sound engineer I find it difficult to agree with your stance about this odd entity 'the music' - every stage of the process should be as flat as possible unless it is an artistic decision to change it. So if I'm recording a live event I should use the best mics, with the flatest response, use the recording device with the flatest response on most headroom, and then master the recording. Now at this stage I can play around with the EQ on the recording and make an artistic decision on the timbre and tone of the sound - because I have not predisposed myself one way or the other by colouring the sound I recorded. I don't disagree that a doctored sound might sound better, but it is not more accurate.

In the real world systems aren't perfect, and those that are close cost a lot of money. Now you have to make a decision of what makes the best sense with your budget. Now some mics and recording systems colour the sound in a pleasing and predicateble way - it sounds like the setup you settled on does. A lot of people forget that the post production of a recording or the setup of the PA at live gigs is a very important part of the music creation process, guitars drums and keyboards may be your instruments of choice, but for a sound engineer the instruments of choice are mics gates EQs compressors and sound desks - in producing a recorded work both the musicians and engineers are important - would the Beatles work have been the same if it hadn't been for the creativity of the Abbey Road engineers who broke from the tradition of 'perfect reproduction' and started working with the musicians to create new ways of presenting the sound - probably not.

In your example the rolloff at high frequency is a common effect with high volume PAs - at high SPLs your ears get tired and the high frequencies are affected first. Most people can relate to that slightly muted feeling to thier hearing after a particularly good gig - so the slightly muted nature of the mic that you use matches people recollection of live gigs. Interestingly popular mics for live work will not be the same as those for live work - even with the same instrument and musical style.

Re:That's all very well but

A decent, but simplistic article. Unless you're a fussy audiophile, this analysis should be sufficient for you.

[rant] I wish the author would present his graphs in a more readable way. A screen dump of Photoshop in WinXP is not a professional way to show data. It's ironic that while reviewing lossy audio formats he opts to use a lossy image format (JPEG) for the graphs. I had to double their size on my screen just to make some sense out of them. [/rant]

It's not difficult to gain better-than-CD quality. CDs have been around since the early 1980s, and their main drawback is that they have a low sample rate, 44.1KHz. This is why many sound engineers prefer vinyl. because it's an analogue format, vinyl has a potentially infinite sample frequency range (although it's obviously limited by the recording and playback equipment, and by the physics of the media itself). Apple has used original masters (not CDs) to create much of its song library, so all they have to do is encode at a higher frequency than 44.1KHz. At a guess, they're probably using 48KHz, which is on par with DAT and MiniDisc.

I'm not surprised that Apple is using AAC. For one thing, it is clearly better than the decade-old MP3 format in all respects, and the licensing costs are probably the same or better. Technically, it may not be as good as Ogg, but most people don't even know what Ogg is so it doesn't matter. As long as Apple can say "our format is better than MP3 and CD audio" (the two prevailing formats), they will have the attention of consumers. AAC is a more mature format than Ogg (Ogg isn't bad, but AAC is more tried-and-proven), and is probably more compatible with existing DRM technologies. DRM is important to keep the recording companies happy and to ensure that the files will only play on devices that Apple specifies (like on Macs and iPods).

A major stumbling block for Ogg is that until fairly recently it was necessary to use a floating point processor to play the format. In the arena of portable devices, only PDAs have floating point capability, which is why you can play Ogg files on your Zaurus and not on your iPod. AAC is already supported by many devices, so Apple has a larger potential market (although at present only iPods can play the files).

Re:Spectrum analysis is useless

According to this [infoanarchy.org] blind listening test conducted by c't [heise.de] magazine, AAC at 128kbps was ranked the lowest of all codecs sampled at that bitrate (WAV, OGG, WMA, RA, MP3Pro and MP3)... One can always hope that the claims of Apple making their AACs directly from the record masters are true, as this would help the situation some.

Re:Spectrum analysis is useless

How can this parent be +5 insightful? It is wrong and uninformative.

I worked with MPEG4 (AAC) and OGG a lot (for my phd. thesis) and spectral analysis IS very important. Although it is correct that it doesn't show precisely what information is left out because of what our hearing system doesn't register. However, these hearing curves and integration times are already known (although not the same for evry human) and most post-MP3 encoders do this rather correct. Most profit nowadays is in clever signal processing. The spectrum of a decoded signal shows almost all artifacts very well and is therefore something which helps a lot in showing artifacts in a coding scheme.

Of course listening test must also be done. They show that modern encoders make choices (not all our ears are the same, and so isn't all the music) which very often pays of in a certain test.

Theoratically AAC and OGG are rather similar, but AAC has a few nice extra's like the Temporal Noise Shaper. However in practice OGG seems good enough (unless MP3) and is free, while AAC is not that much better and unfree, so my choice is obvious.

I will wait for the OGG hack of the IPod, now it had a better processor.

They chose AAC because it's already in QuickTime

And it's more efficient than MP3.

Their encoder is not particularly good, and AAC is covered by a ton of patents, so there probably are other reasons why they chose it.

For anyone else but Apple I see no reason to use AAC when you can have Ogg Vorbis.

PS: Shameless plug: I wrote a vorbis patch to add SSE support [www.fefe.de] for enhanced encoder and decoder speed. It also contains some 3dnow! optimization for you K6 users, decoder only.

AAC is pretty weak, no marketing can change that.

And it's more efficient than MP3

At low bitrates, AAC is very weak, at 128kbps it was the worst of all:

Study [infoanarchy.org]

I was one of the 3000 participants, btw. And my ranking which I gave (blind, I did not know which sample was which) confirms pretty much the results, at 64kbps, AAC was unbearable, while ogg was not distinguishable (by me anyway) to the original.

The only test where AAC didn't fail miserably was the "expert test" with only 8 listeners.

OGG has beaten all other codecs consitently at all bitrates.