Does Portable Music Have to be Compressed?

FunkeyMonk writes "The Christian Science monitor has an article discussing the gap between music fans and audiophiles when it comes to portable music. Would you pay a few cents more to have lossless downloads from iTunes and other online music retailers? As a classical musician myself, I choose not to download most of my music, but rather rip it myself in lossless format."

Lossless is compressed

...just with no quality loss. Perhaps the question is "Does portable music have to be lossy?"

Re:Lossless is compressed

Ahem, http://flac.sf.net/ [sf.net]

A used for Magnatune downloads (among others), and supported by decent media player software and a handful of MP3 players

Re:Lossless is compressed

I'd start by massively oversampling the data at initial recording time. 100 MHz, 26 bits per sample, 8 channels, etc. Something totally outrageous and utterly unusable for anything resembling sane. The next step is to split the sound up NOT according to source or frequency, but according to what groups together the best. Here's where the oversampling comes in - you don't NEED the actual data points to get lossless encoding, you only need to be able to recreate them. Thus, once we have grouped the compressable information, anything that is left over that can be reconstructed is of no further interest and we can ignore it. The same goes for any complete grouping that we have formed - if the complete group can be synthesized directly from one or more other groups, we don't need it. You then compress the groups - in isolation or as a simultaneous set of systems - either losslessly or using a lossy method. When you downsample, you eliminate the guesses that are wrong first and then eliminate duplicate guesses that are right between the groups.

What you will end up with is some set on N systems, which will be large amounts of noise with small amounts of useful sound in them, which when superimposed with each other AND a filter function produce the original sound and which when taken individually are highly compressable. (The noise is simply there to create fake patterns that we can compress. It won't be random noise, because that doesn't compress, but is noise in the sense that it has no meaning or purpose other than to produce nice mathematical functions. The filter is simply something that's used to extract this deliberately injected deluge, so that the output is valid.)

Is this a valid technique? Well, yes - it's not that unusual to add noise to simplify compression, then subtract the noise afterwards. That's fairly standard. Splitting the data up to simplify the noise is merely a variant on the idea, and is used in plenty of compression methods. Compressing individually seems to be the customary method, but computing power is more than adequate these days to use fancier techniques IF justified. (Since you can encode the decoding method at the start of any track, it should be wholly irrelevant as to what method is used, provided the computing power is there to run it in real-time.)

Re:Lossless is compressed

"But last time I checked, all sampled music was PCM, and that's lossy by definition."

Your ears are pretty lossy too. Anything recorded using 192kHz/24bit has more dynamic range and a lower noise floor than your ears do... not to mention the fact that no equipment exists that can do better than ~100dB of dynamic range, and SNR of <0.1%

Re:Lossless is compressed

If the music is digital, it is by it's very nature lossy. To convert sound into a digital format, you must sample it. No matter how small your sample, there are gaps between them. The gaps are lost when you digitize the music.

But yeah, from a digital perspective, things can be compressed such that the original is reproducible ("lossless") or an approximation is reproducible ("lossy").

Layne

Re:Lossless is compressed

Let's not perpetrate the myth that music can be recorded losslessly in the first place. All sampling is lossy. CDDA specifies a certain sample rate, beyond which you lose higher frequencies, and a fixed number of bits per sample, so you lose precision. For the same bitrate, you would get better results by starting with a high-resolution master and using lossy compression down to CDDA bitrate.

I'm not arguing that a lossy encoding of CDDA is as good as CDDA; it isn't. Just that there's no law of nature establishing CDDA as the gold standard in the first place.

Re:Lossless is compressed

Just that there's no law of nature establishing CDDA as the gold standard in the first place.

There is, however, a rule of market that establish the CD as the only source users can encode their music from.

Re:Speaking without detail is useless.

You're right, but again we have to consider what people are able to hear. The number of quantization levels that are used insures that the human ear can't tell the difference between two intermediate levels.

It's funny, I have an audiophile acquaintance who swears that records are superior in every way to "digital," and for the same reasons described above. The funny thing is, because of the large number of quantization levels used in a CD, the CD's dynamic range far surpasses that of any record player. More info here [georgegraham.com]

Theoretically, yes, analog would always be superior. But in reality, physical limitations of the stylus on a record player limit that medium far more than quantization limits the CD. Those same physical limits exist in the human ear, too.

So, while digital might not be "perfect" theoretically, it's "perfect enough" allowing for the limitations of the human ear.

more for non-DRM

Actually I'd like to be able to get an "original" image a la the CDs you buy, but allow single CD tracks. Would I pay more for that? I don't know. I've never bought any of the DRM'ed crap because it's DRM'ed, so I don't know how badly (or well) compressed they are.

If there are audible compression artifacts anywhere in today's downloadable DRM'ed music I'd probably insist the compression be less or not at all, after all I'm paying for music, and a compression artifact (to me) is analogous to stuck pixels in a monitor or camera... my threshold of tolerance is zero for that.

(I had one of the very original SONY Mini-disk recorders, and remember a passage of a Doobie Brothers track where some high pitched bells instead of sounding like high pitched bells sounded like someone sneezing... unacceptable... completely altered my experience of MD (along with numerous other things about SONY).)

So, bottom line, DRM aside, I consider it the responsibility of the music industry to deliver what they claim they are delivering... music (usually). I'm willing to bet what they are delivering has artifacts... I wouldn't pay more to get rid of that, I'd demand they replace the defective product.

The nice thing about my CDs and my derivative mp3 collection (recorded at 320 VBR) is if I hear an artifact in my track, I have the unedited original, I rip it at higher quality until the artifact isn't there.

(As an aside, I think the article makes an exceptionally great point not directly related to the users:

That's important to sound engineers, too. "You spend a long time training your ears and striving to perfect your craft and put out a better product," says Jeff Willens, an audio-restoration specialist at Vidipax in Long Island City, N.Y. "When you finally discover that these things are being listened to on cellphones and through pea-size earphones, it's kind of disheartening."

So, in addition to short-shrifting consumers with less-than-perfect (to the ear) product, the movers of downloadable music thumb their noses at the collective profession of sound engineers and engineering... pretty rude.

Granted, a lot of the music out there is crap -- it's no justification for compromise on the medium.

Oh, and re the subject line of my post... I'd pay a little more for non-DRMed music, not uncompressed music.

Re:more for non-DRM

So, in addition to short-shrifting consumers with less-than-perfect (to the ear) product, the movers of downloadable music thumb their noses at the collective profession of sound engineers and engineering... pretty rude.

Not all sound engineers are as dedicated to the art as you suggest. Okay, sure, if one wants to listen to something recorded in a state-of-the-art lab by consummate lovers of both the music itself and clean audio in general, then one should invest in the right conditions.

From my own collection, I'll take the world premiere recording of Boulez's Repons [amazon.com] as an example. It was recorded in the projection space at IRCAM, one of the world's foremost music and acoustics research laboratories, and I only listen to it from the CD on my home stereo system, which isn't the most whizbang, but the best I can afford.

Contrast this with Rush's 2002 album Vapor Trails [amazon.com] , a musically strong release which was recorded in poor circumstances and remastered in worse. The clipping that plagues every track in the album has long been criticized by fans (see the Amazon reviews for further info). So, since the guys who engineered the album didn't aim for clear audio, I feel no shame in putting this in 160 kbps Ogg Vorbis and listening to it with merely average headphones on my portable MP3 player.

As has already been said in many places in the discussion, lossless is probably going to be a draw mostly for classical (or, in my case, modern-classical) fans.

Observation on music quality

I'm a musician in my spare time, and pretty much have noticed that other musicians really don't care much as to the Quality of the recording that they listen to. Recording songs in a studio is an exception, but what I've noticed is that many of my friends just listen to their music on crappy boom boxes, etc. Is it a function of being poor - nope haven't seen that. But what I have noticed is that a majority of "audiophiles" are not musicians. Yes, of course we'll see the few exceptions, to prove a point, but generally musicians are interested in the chord progression, melody, rhythm, instrumentation, etc. The recording quality is the last thing we care about when listening to a song.

Re:FFS shut up already

Yes, there will be a quality diff between #4 and #1, but it'll be the same miniscule PSNR loss as from #1 to #2. So unless you transcode a dozen times or something it won't really hurt you.

Every encoder will generate ringing and other artifacts. Every good encoder tries to put those artifacts just a bit below the hearing threshold according to an algorithm that has been tested extensively with normal music. However, encoders are generally not fine-tuned to deal with the unnatural type of noise that results from another encoding process, resulting in the noise ending up above the hearing threshold after the second time.

You might wish to check some double-blind test results on HydrogenAudio [hydrogenaudio.org]. Short version: reencoding 256 kbps MP3 to 128 kbps MP3 sounds horrible compared to 128 kbps MP3 straight from the lossless source.

Re:FFS shut up already

It does mask (cut off) but it's based on the psychoacoustic modelling

Yes, they are based on psychoacoustic modelling. But I believe that it is mostly a few curves that define the hearing threshold for certain frequencies in the presence of a loud masking tone. The rest is trial and error, with lots of fine-tuning of a zillion parameters in the algorithm while listening to compressed music and asking the golden ears at hydrogenaudio to compare different versions of a codec (at least for the OSS ones). There is no algorithm that will give you the degree of transparency of an encoding as a number that realiably matches the results of double-blind trials.

Regarding generative losses of enoding: masking can for example be done by using the fact that a listener doesn't hear pre-echoes before sharp attacks as long as they don't come earlier than X milliseconds before. The encoder uses this fact to get the bitrate of sharp attacks down. But on the second encoding, the pre-echo might become 2X milliseconds rather than X milliseconds, and be audible.

I've seen reports on hydrogenaudio that codecs such as LAME that use complex psya modelling extensively do a worse job as a source for transcoding than fast high-bitrate codecs that have much simpler algorithms for throwing away information.

So it's entirely possible to make a lossy codec which loses NO information through further encodings (though what would be the purpose?).

I suppose you are talking about transcoding to the same bitrate MP3 with the same psychoacoustic model. That could be useful if you want to get rid of DRM by burning to CD and then re-ripping. But the question is whether transcoding to a lower bitrate or even different codec will give audibly different results from encoding directly from the source.

Re:FFS shut up already

Audiophiles have consistently been failing double blind tests [wikipedia.org] when it comes to lossy vs lossless audio compression.

Now, if you wish to sell stuff to audiophiles, then players supporting lossless compression are excellent - they will buy it (along with anything you claim, on whatever grounds, will improve the playback quality).

If you however want to bring better music quality to the general population - make them get better headphones.

Re:FFS shut up already

I'm working on a product right now called a Sound Squid. Through years of intensive research (unsupported by double-blind tests of course! People who aren't audiophiles are human trash and shouldn't be allowed to touch my $5000 headphones!) I've discovered that the common squid, by using its various air pouches, tentacle positioning, and movements of its staring eye can enhance any listening experience.

Your Sound Squid kit comes with:
1 (One) 10 gallon aquarium
1 (One) squid, family Loliginidae
8 (Eight) water-proofed high quality tentacle clips
1 (One) computer-controlled Sound Squid->digital interface
1 (One) instruction manual in English, Japanese and French

All for the low, low price of only $7,500!

You may also be interested in the following accessories:
Gold-plated Monster brand tentacle clips ($1500)
1 week worth of squid food ($350)

Can you afford NOT to buy a Sound Squid?

Re:FFS shut up already[FORMATTED]

Finally someone who makes some sense in this discussion. I agree with most points you made (I will disagree with one a bit further down). "One cannot hear a difference" is one of the most annoying /. memes to me, regardless of whether it is applied to lossy codecs or good audio equipment in general.

About lossyness:
I agree with you that ears can be trained, and that you won't miss stuff if you don't know it should be there in the first place, or don't care whether it is. When I decided how I want to encode my music I did a quite extensive test and I found that to me even high-bitrate mp3 encodings made by lame can sound noticeably different from the CD. For example, I encoded the first track of Mike Watt's Contemplating the Engine Room [amazon.com] CD. It starts with an e-bass solo, and using reasonable lame presets there were no artifacts and I certainly could hear the notes played. Somebody expecting nothing more will probably be happy with the compressed sound. However when you know how a bass can sound and listen to the CD, you realize that there is so much more in Watt's bass sound: it is full of harmonics that make the bass come alive and turn it into the recognizable Watt bass in the first place. And these harmonics are gone even in the highest lame preset. (And oggenc adds a nasty hiss which makes the song completely unlistenable.)

About equipment:
You said "All of the high end audio products generally have no benifit for the average consumer, but in a studio setting, when trained ears are listening, that expensive gear tends to be more valued", and that's where I disagree a bit because you make it sound as if only a professional sound person could appreciate good gear. I's agree that someone who is not particularly interested in music has no need for good gear. That's pretty obvious. If you're going to listen to music only as background noise while cooking, go with the cheap stuff by all means.
However I would argue that everyone who likes music and spends time actually listening to it will profit from good gear. To everyone who doubts that I can just recommend to grab a few favorite CDs and make an appointment at a good hifi shop for a listening session. "Good" means "a shop that has solid equipment from the lower to very high price ranges, but that will not rip you off by trying to sell you air conditioners."

Not directed at you, but I need to say this once on /. because it has been bugging me a long time:
To those discussion contributors who lose all ability to differentiate when they hear the word "audiophile": one cannot deny that wackos exist in this field. On the other hand, since when is being an analog geek not allowed on /. anymore? Sound recording and reproduction (that is, turning a complex air vibration into an electric current, storing it in some form, and later turning it back into an air vibration again that sounds as close to the original as possible despite this happening in a completely different room situation) is an extremely complex topic. And, like it or not, there is still a significant analogue part to this, and will be for the foreseeable future. This means that you have to live with the difficulty of interacting with the real world in a less deterministic way. Only recently has it become possible to simulate microphones, amps, and speakers digitally, and sound reproduction has benefited tremendously, especially by making good gear much cheaper. But until then the only way to become better was to design analogue gear and try it out, relying on basic measuring equipment and your ears to assess the sound quality. This IMHO is hardcore geekdom worthy of honorable mention on /. and not ridicule. There were and are serious practitioners out there like Nelson Pass [passlabs.com] or the naim guys [naim-audio.com] who have dedicated d

Re:FFS shut up already

I'm with you on this one. Even CDs are inferior to studio tape when I listen to them on my magnetically levitated speakers hooked up to the stereo with gold-plated cables. The tape heads are gold-plated too. The whole setup sits atop a few tons of sand bags and is located in an underground chamber enclosed in a steel Faraday cage.

it depends

It depends on how you intend to listen to your music. If you're going to be listening to earbuds while you're outside or working out at the gym or whatever, then compressed files are fine. Enough ambient noise will be getting through that you'll barely notice any compression artifacts, if at all. However, if you intend to listen to music through a nice set of headphones or speakers in a quiet listening environment, then you'll want it to be as uncompressed as possible. The same generally applies for music with wide dynamic ranges, such as classical/orchestral music.

What's the point?

What's the point? The bottle neck on MP3 players is not the audio files but the decoding/playback hardware and even more important the headphones. You simply can't hear the difference after a certain MP3 bitrate like you can on real audio systems with proper equipment.

Whenever I buy a new MP3 player I spend a few minutes to find the sweet spot where I simply can't hear any difference with a higher bit rate let alone lossless audio. This is almost always 128 kbps, even with quite good head phones.

I would pay a few cents less

I would personally pay a few cents less to get CD Quality music. Often when I buy CDs they are priced anywhere from 7.99 to 13.99. I think that if you average it out, the CD ends up being about the same price as iTunes, possibly a dollar or two more. But for that extra dollar, you get a physical copy, that's lossless, and doesn't contain any DRM. I try not to buy CDs with copy protection, and even for the few I do, I can still easily rip them, by disabling autorun. The only advantages of iTunes and other music services are, the ability to buy one track, and the ability to have it right away. I don't usually buy music from artists who can't fill up a whole CD with good music, and I'm not that impatient that I can't wait for the CD to arrive from Amazon, or wait until the next time I happen to be in the mall. Sometimes, if I know I won't be in the mall for a while, I'll download the cd in MP3 format and then buy it later. So, I could buy off iTunes, but i'd get music that was of inferior quality, and locked by Apple, which means that I couldn't play it on another MP3 player without degrading the quality even further.

GIGO -- Garbage In, Garbage Out

Well, the poster of this article obviously doesn't consider CD quality to be "lossless." How far we've come from the OLD audiophiles who wouldn't touch anything that wasn't a meticulously cared for LP -- or better yet, reel-to-reel tape in your home rig.

How much longer before we consider 128-kpbs MP3's to be the "standard" for quality music, especially as we're moving to more and more of a "download on demand" compression crazed society?

Won't anyone think of the children!

Re:GIGO -- Garbage In, Garbage Out

If the sales numbers are right for XM radio and Sirius radio, 64Kbps will become acceptable.

Both the sattelite radio services have incredibly horrid sound. anythign with high frequencies has twinkle and other nasty artifacts that are so prevalent it renders it unlistenable to most people who like clear music. I have went back to FM at times because Sirius and XM suck so bad.

Now we have robot radio stations around here that are mp3 based and LOW bitrate mp3 based at that. My daughter was listening to one of them and I asked, "when did you get a XM raio in your room?" she let me know she was listening to the new Rock FM station.

Current state of music is swirling the toilet. I havent heard a decently mastered CD in decades, radio and supposed "CD QUALITY" Digital FM and Sattelite all sounds worse than 128kbps mp3's on a $6.00 mp3 player.

All around the music quality stinks. Even if I could buy a uncompressed high bitrate version, the mastering at the studios is so sub par it wouldn matter.

Doubt it will happen

I doubt any representative of the RIAA could keep their blood pressure down with the words 'losslessly reproduceable content' and 'internet' in the same sentence. Given the disputes over uniform music cost and how much they resisted distributing even lossy DRM'd audio in the first place, what are the odds we'll see this?

Re:We need a new Hi-Def Audio format

I remember talking about MP3s with an audio engineer friend about a decade ago. As an engineer, he said that he would prefer MP3s to be mastered for the format, which means any limitations of the MP3 and other compressed file formats would be taken into account to minimize/delete any perceivable quality loss. For instance, the cassette version of a recording is mastered differently from the CD version, since tape has different audio qualities (the same also applies for vinyl versions). They don't just stick the CD master onto cassette tapes. On this point, I fully agree with him. However, it seems that all of the AAC/MP3/WMA files that you can buy are sourced from CDs, rather than being mixed especially for the format.