Specialized MP3 Compression Hardware? 26
Syn Ack asks: "Does there exist any specialized hardware for the sole purpose of compressing audio into MP3 format? Without going into too much detail, I'm putting together a platform that has to rip and compress as many songs as possible in the shortest amount of time (ie compressing several thousand CDs). I was considering a large ~20 machine *BSD or Linux cluster of high end x86 or Alpha hardware solely for compressing digital audio into MP3 format. Before I go ahead and look at other solutions, I wanted to check the distributed minds of the Slashdot readers to see if they know of any specialized hardware for this purpose. Price isn't really a concern. Something like the dedicated encryption boxes/cards used on Web servers is the type of thing I'm thinking of. Anyone have any ideas?"
recommendation for cdrom drives (Score:2)
dunno about hardware, but why not put up 10-20 boxes with the above drive, or maybe sharing a tower to make cd-changing easy?
Re:Sounds like (Score:1)
Malk-a-mite
hardware, or targeted compile options? (Score:3)
Also, if price really is no object, for efficiency's sake, rip them all to wav first, then encode later.
If quality is a concern, use something like EAC (Exact Audio Copy - for Windows. There's a Linux/unix equivalent available, but I can't remember the name right now) when ripping. You'll get copies as accurate as possible before you start encoding.
Make sure you use the right encoder, and make DAMN sure you use the right settings!
LAME is considered the best encoder by most people, especially at higher bitrates. (I wouldn't bother with anything less than 128, but for my own collection, I do 256.)
Re:hardware, or targeted compile options? (Score:1)
Steve.
use DSP to offload server processor (Score:1)
Haven't heard... (Score:1)
I haven't heard of anything that would be directly applicable to your situation.
I'm sure that there are DSPs that work well for encoding MP3s in real time, intended for MP3 walkmen and stuff.
But you are talking about really cranking them out at faster-than-real-time. Which probably means you want a bank of drives and a traditional PC cluster with whatever CPU will encode fastest. My feeling is that someting with 3DNow/KNI would work better, but I'm not sure.
Besides, general purpose computing is better. If you don't need all of the machines, you can put them to work for other tasks.
Re:Sounds like (Score:1)
sblive mp3+ (Score:2)
The SoundBlaster Live! mp3+ was touted as having onboard mp3 encoding capability (through the firmware...it's the same card as the others). Specifically, it claimed a 5x acceleration up to 320kbps. As far as I can tell (I've got one), the acceleration is only used by the included mp3 encoding software.
The thing is, with that and a p3-500 I can turn an album's worth of wav's into mp3s in about, oh, 10 minutes (vs. about 50 minutes in software using BladeEnc).
I've got a friend with an 850MHz Athlon and a Kenwood TrueX 72X drive. With the Xing encoder he can do an album in 5 minutes flat...from the CD! The 72X drives are a MUST for mp3 encoding...the drive speed will almost certainly be your bottleneck.
lame, and distributed encoding.... (Score:1)
Some automation (Score:1)
Perhaps you can have a dedicated CD tower with 15 or so drives (SCSI, of course) and have the robot drop a new CD into whichever drive tray happens to be open, and press the drive closed.
Have the CD eject when done, the robot's touch sensor rigged to 'see' this and pop a new CD in. I think it might be worth the effort - you'll only have to keep the robot 'fed' to keep the encoding going!
Naturally this would help - if you want to do the encoding quickly you'll have to use your hardware 24/7. This is one way to keep it going. At the very least, it's a neat excuse to play with Lego!
Hardware? Kind of... (Score:2)
Sigma Designs [sigmadesigns.com]
Darim [darvision.com]
Optibase [optibase.com]
Good luck.
Use commidity stuff.. (Score:2)
1000 CD's would take two weeks, granted, but extrapolate the numbers. For $200 you can purchase a board/cpu combo 1.5 times as fast, and handle that load in just over nine days. By the time you spend $1000, you have enough processing power to handle all those CDs in 45 hours. What about ripping? Well, using the best CD ripper for Linux gives me about 1/4-1/2 the rated max speed of the drive. So, to feed each 'node' (This is batch processing, so the term is in quotes) at the rate it consumes wavs you would require at least one 40x CDROM drive. Another $80 should cover that. Need Ethernet and cases, so toss on some $25 Tulip cards and a cheapo $40 case.
We've spent $1,725.
Okay, to cover the final costs; Three 40G IDE drives should handle the compressed audio and the temp space required by the compression farm. Say $600 for those drives, another $500 for the server they reside in, and $200 in associated 100T network cost.
We're up to $3,025.
Whether to place the CDROM drives in the server or in the nodes is left as an exercise for you; Putting them in the server is more convenient, but you'd require $100 in SCSI hardware. Doubling the number of CDROM drives doing the rip will decrease the number of trips you have to make to change discs to every half hour, but at that point it is vastly less of a headache to go with a pair of 14 disc cabinets and change every 1.5 hours. Really pricey to do CD cabinets tho..
So, $3K for 500 CDs/day the cheap way, add $600 for some extra CDROM drives and SCSI hardware to make your life easier.
Oh. Everything assumes you've got someone changing discs 24/7. Tweak for the desired work day.
Someone else mentioned EAC for Windows; The equivalent for Linux/*BSD is called CD-Paranoia.
CD Rom Drives (Score:2)
Using this CD-ROM, the bottleneck on my p3-800 system is the encoding itself. It takes only 4 or 5 seconds to rip a 20 minute song, but more than a minute to encode it. This was your original question, though
I've been using the AudioCatalyst product for about two years now, and it has been (and continues to be) the fastest and best sounding (don't bore me with comparisons of audio plots -- it's not the data that's in the MP3 that matters, it's how it sounds) MP3 ripper/encoder that I've found. CDex (http://www.surf.to/cdex [http]) is a close second since it fits nicely on top of any encoder that you want. I've gotten comprable speed out of Cdex as from AudioCatalyst (http://www.xingtech.com [xingtech.com])
Re:hardware, or targeted compile options? (Score:2)
-----
Re:Sounds like (Score:1)
anacron.
Re:Some automation (Score:1)
Looks like DSP chips are in (Score:3)
It looks like PCs are hard to beat. Here's a claim of a Fraunhofer encoder at 10X real time on a 500 MHz Pentium3 [216.205.158.3]. Maybe it's vapor, but if it's real it's certainly a lot faster than LAME 3.87 (beta) [sulaco.org] when I tried it on a 800 MHz machine (Lame ran at approx 2X using "-V 3"). I recall hearing claims of LAME doing real time on a 266 MHz PC... maybe it does with different settings.
Constraints (Score:4)
Since a single load-person could likely service two readers alternately this would make a work station. This work station would then produce 2 CD-datasets every 5 minutes. Assuming about six hours of loading per day one would achieve around 144 CD's of data loaded per station per day.
Assuming two CPU's per station (1 CD-ROM/1 CPU) one would have 20 minutes to process each CD over 24 hours in order to keep pace. As I believe this rate is achieviable with a current fast system then you have a well tuned solution.
Workstations consisting of 1 load person with two fast PC's = ~140 CD's per day processed
No need for custom hardware, no need for clustering, all off-the-shelf stuff.
Kenwood 72x problem? (Score:1)
No hardware, but try software (Score:1)
Wrong question. (Score:1)
Unless your use is tied to current MP3 technology ONLY, you're being very short-sighted by making MP3 compression your primary concern.
Your concern should be about how you can read and store as many CDs in raw format so you're always ready to convert the library to the current audio format, be it MP3, Ogg Vorbis, WMF or whatever else tomorrow's standard may become.
Re:Operating System (Score:1)
Something that might help, if it exists? (Score:2)
Re:Looks like DSP chips are in (Score:1)
Get a good Blade front-end, and even a decently fast machine, and you can do pretty fast encodes. Doing 320kbps with Blade on my Tbird 900 under Win2k Adv Server goes up to 4.6x, if I'm not running much else, and dips to around 3.95x if I'm using the computer heavily.
I'm not sure if things like the P4's SSE2 would help, but seeing as how an optimized version rips the Athlon to shreds in Flask MPEG4, (see Tom's [tomshardware.com] perhaps MP3 audio is similar enough that it could also take advantage of SSE2 to great benefit...
Re:Wrong question. (Score:1)
I had limited space to elaborate on my questions. You are correct that storing the audio in a raw uncompressed format is the way to go. Initially we won't be doing this soley for the added complexity however we do expect to add this as an enhancement to our platform later so that we can convert the uncompressed audio to any compression type at a later date.
Paul
Re:No hardware, but try software (Score:1)
Yes, we have looked a that type of solution. I have, through my limited tests, come to the realization that RIPing the audio is a slower part of the work than the actual compression. I've tried RIPing on both a DVD and regular IDE CDROM drive and they both took way more time to RIP the audio than the computer took to compress it. So perhaps my concerns in compression are unfounded and I should be looking for a very fast CDROM drive that can RIP audio the best.