Slashdot is powered by your submissions, so send in your scoop

 


Forgot your password?
Close
typodupeerror
×
Hardware

What File System For Portable MP3 Player? 23

Posted by Cliff from the digging-into-the-internals dept.
Komi asks: "I am a senior EE student at Purdue, and in my design class we're making an mp3 player that can read from a hard disk. We're using an 8-bit microcontroller (Rabbit 2000), and I've read documentation on how to interface it with a 16-bit IDE interface (where you latch the upper bits and get those on the next cycle). So actually doing reads and writes shouldn't be a problem. My question is what file system should I use? I want to be able to copy the songs to the hard drive from a PC, then attach the drive to my player and have the player read it. I've read that the FAT system is a horror to use on a small micro, so I was wondering if I should use ext2 [?] or ext3 [?] . Or should I just stick to reading from CD's?"
This discussion has been archived. No new comments can be posted.

What File System for Portable MP3 Player? More

What File System for Portable MP3 Player?

Comments Filter:

  • Brownout - power failure during any phase of operation. This includes especially writing operations.

    You combat this by having simple chkdsk-like functions which just ignore errors (flag them, free the sectors allocated without directory entries, whatever) -- remember you're storing MP3s on here, not term papers. MP3s can be replaced so this isn't really all THAT critical if music starts disappearing, so long as the rest of the songs are unaffected.

    FAT even worse (deadly self-crippling easily possible).

    As I said in an earlier post... you guys are going WAY overboard here. VFAT (long filename extensions on top of FAT) is more than adequate. You don't need to worry about fragmentation, you don't need to worry about packing tails, you don't need to worry about journalling. It's an MP3 player, not a file server.

  • I have encountered FS that were unwriteable or even completely unusuable after a serious crash/problem. THIS is the problem I see.

    That's because of the fs drivers in the OS; not an inherent fault in the fs itself. Personally I have never seen a FAT[16|32|VFAT] system die as horribly as you've described and I have been using these filesystems since the DOS 3 days. Mind you I've seen some nasty corruption but almost always restoring from the 2nd copy of the FAT has been successful in gaining access to the data (or at least the vast majority of it)

    You're right; people will not accept this in an appliance. However with a system mounted RO 99% of the time I really don't see this happenning.

  • would recommend a journalling file system, because you don't want to lose your MP3 selection if you don't unmount your drive properly (which could happen otherwise).

    Don't do much real-world design I take it...

    Why not just use FAT32, it's an extraordinarily simple fs, you can put the drive in a plain-jane computer and get data on and off it for debugging the FS if need be. To combat shutdown problems, just mount the damn thing Read-Only unless you're going to be putting data on it. There is absolutely no reason to keep the entire fs in a read-write state. The playlist can be brought into memory (simple index) and any changes to it can be made permanent via a "commit" button which remounts rw, updates and remounts ro again.

    Hell you don't even need a "commit" button -- when the device gets hooked up to change the fs you could commit the changes then. The amount of data which changes on an MP3 player is minimal.

    Also, although you don't say it's portable, if it is, a normal hard drive won't be suitable since they won't stand the movement. If it is portable, use a solid-state device.

    With drives as cheap as they are and as rugged as they are, you should have no problems using desktop-grade hard drives in a portable environment. No, they won't last forever. Laptop drives are getting pretty cheap too and they will last quite a bit longer but saying that desktop-grade equipment won't last is deceptive at best.

    Finally, as to hard drive vs. cd, cd is much more flexible, since you imply that the device is a music player in function and hence there won't be any advantage from being able to download music onto the hard drive, and music players aren't much good unless you can change the album.

    I agree with you here; either give the player a network jack and use SMB (nothing would be cooler to the class than plugging it into the network and just copying stuff on and off it); you don't want to have to pull the drive out and hook it up to a computer. USB may be nice too, depending on how you want it to work.

    MP3 players I've concieved worked off the simple principle that there was an ATAPI interface and whatever you happened to hook up (generic HDD, CD, DVD or CF (in IDE mode)) the player would have to have drivers for. That way you have some choice in what to use for media while the base software stays the same.

  • microcontroller plus hardware decoder
  • The problem I see with FS problems (e.g. after brownout) is not the data loss itself. There I am with you. I have encountered FS that were unwriteable or even completely unusuable after a serious crash/problem. THIS is the problem I see.

    So if you use simplistics (V)FAT you need an idiot-proof method to restore the FS (functionality) itself. While this could be done with a (possibly automatic) "format disk" function, I think users won't accept something like this on an appliance, even if they are used to problems like this on a PC.
  • Before you chosse the filesystem decide what you want.

    If you opt for CD-ROM (iso9660) you do not have to care for brownouts and power-off "shutdown"s. You will not have to care for write routines - and have the plain audio-cd output as free bonus.

    If you want a re-writeable MP3 player, you will have to take the following things into account:
    • Brownout - power failure during any phase of operation. This includes especially writing operations.
    • Small size/performance requirements - XFS may be a bit of overhead for the small CPU. What about the other FS?
    • Self-repair - if an(y) error occurs the user cannot interact due to a missing console. he FS will have to be corrected automagically without data loss!

    So ext2 is a bad choice (missing journal/self-repair), FAT even worse (deadly self-crippling easily possible). ext3/ReiserFS might be an option, but I do not have personal experiences with that yet.
  • Assuming you have the space in your firmware, a player that reads CDs should be a flexible as possible. Inlcude support for ISO9660, including all the extensions. Include support for HFS, so it can read Mac-only CDs, just to be safe. Include support for UDF (DVD file system), in case you get a chance to toss in DVD-R reading hardware in the next revision. You could also look into other file systems that might make sense (like ROMFS); there might be some with less overhead than ISO9660 that would work just fine.

    And you didn't ask, but you should add support for ID3 tags (revisions 1.0, 1.1, and 2.0, if I recall correctly).

    And survey users of other MP3 players to see what they wish their player could do, and make yours do it.
  • That might work, but you probably don't want to do that. There isn't enough redundancy in regular uncompressed CD audio to get away with that for compressed audio. I'm not clear on what VCD tracks look like, though--it may well work. Either way, though, you probably don't want to use over 100 tracks--those standards usually aren't designed with such a large number in mind, so I don't know how well they would deal with it.

    Still, that's an interesting idea to look at.

    Also, with the proliferation of various CD players that will play MP3s, we should be working towards a standard format that they will all play. Using ISO9660 seems to be that format, at least for now. Any player you build should start with that as a base, and then add as much more flexibility as possible, given the limitations of the firmware size.

  • OR... (Score:1)

    by schon ( 31600 )
    I would recommend a journalling file system, because you don't want to lose your MP3 selection if you don't unmount your drive properly (which could happen otherwise)

    Or he could not bother screwing around with this hassle and use any other non-journaling FS, and turn off the disk cache by using the 'sync' mount option.

    Think about it: the disk in an MP3 player is gonna be doing one of two things at one time: reading (playing the file) or writing (saving the file).. it won't be doing both at once, so the performance of benefit of a disk cache would be negligible.
  • Hmmm.. Mabye you have something, well, aside from the conclusion you need a real file system. He's using an 8-bit microcontroller, and implementing ISO9660 is going to be a pain in the ass. It's a project, not 'production' so the setup time isn't a worry either..

    I say just write the raw concantenated image to the CD, along with a list of file positions. Hell, just write them in raw mode sans list and play them sequentially, tossing in last/next track buttons is easy. The project is hard enough as is..
  • Sorry this is off topic, but I wanted to say it is great to see schools making useful projects.
  • Komi, I hope your MP3 player project goes well. If you haven't already seen Peter Kovac's MP3 Projects Web Site [mp3projects.com], you should certainly check it out.

    For the last several months, I've been working on my own MP3 player project which is similar to yours [pjrc.com]. It was mentioned here on slashdot [slashdot.org] a few months ago. My design uses a 8051 microcontroller to control everything, Xilinx FPGA to move data around quickly, and the STA013 chip to decode the mp3 data. The design uses a 72-pin SIMM to store data, though I'm only just now getting code to effectively manage caching megabytes of data. My code is open-source (GPL), and you can download firmware source code here [pjrc.com]. I'm hoping to make an initial release with the completely re-writen FAT32 and IDE drivers in just a few days.

    I've spent many many long hours and many dozens of all-night sessions hacking on this little project, and I think I can safely say from experience that FAT32 makes sense for a hard drive based MP3 player project. I'll explain, in the context of just a brief history of my filesystem hacking... there's a very detailed project history [pjrc.com] on-line.

    My first attempt at the player was this old design [pjrc.com], which I started in December 1999. You can read about it at the web page, but filesystem wise, it didn't use one. I wrote the MP3 files directly onto the sectors of the drive with a little perl script (redirected to /dev/hda). It actually wrote a little table at the start of the drive (where the boot sector and partition table would normally be) so I could make next/previous track buttons.

    It is very easy to do this non-filesystem approach, so as a student project, you should certainly try to do this first. Many student projects run out of time, so I'd really suggest you do raw sectors and get some sounds coming out, and then if you have time, try for a real filesystem.

    The old design worked pretty well, but it did have limitations. The raw sector approach and perl script meant that all files had to be copied to the drive at once (because the script packs them one after the next.... and the firmware just plays sectors one after the next and doesn't even worry about which ones are what tracks, until you press a button). The 8051 microcontroller wasn't fast enough to play 256 kbps (14.7 MHz, standard slow 1:12 cycle architecture). I've seen several other projects on the net using a 20-some Mhz Dallas high-speed 8051, most notably Anton Verheijen's SoundBastard [hotbot.com] player. I wanted even better performance than that, but at low power with a large DRAM buffer, so that I could play from batteries.

    Well, my new design does it the hard way. I stayed with the slow 8051, and cut the clock speed in half (lower power). I added a Xilinx FPGA chip, which I used to make a custom DRAM and DMA controller. The DMA allows the IDE drive to be read pretty quickly (2.45 Mbyte/sec is damn fast relative to the power used), and the DMA allows large blocks of MP3 data to be sent to the STA013 chip with very low CPU overhead... which is necessary since it's so slow.

    Getting back to the subject at hand, filesystems, I used a very simple firmware approach with this new fancy hardware. The simple (and limited) firmware is still more or less the current rev, but I'm getting darn close to a near total rewrite that will be really cool. But, the point is that there are some simple and not-to-hard ways to read FAT32 filesystems. I'll ramble on a bit about it, even though this post is getting long enough that it probably doesn't all appear unless you click for more.

    The first thing I did to simplify the FAT32 code was to require that the drive be fully defragmented. The thought was that it's not too hard to run defrag before you shut down and pull the drive from your PC. Even if you don't like having to run defrag, consider that it's taken me quite a while (probably longer than you project's deadline) to write the more complex code, so starting with a fully defragmented requirement is probably a good idea. If you have lots of time or you're really good (and fast) you'll always be able to improve the code and remove the requirement for defrag later. (BTW, does anyone know of a linux based FAT32 defrag utility)

    Microsoft's defrag doesn't defragment directories, so you've still got to deal with following the cluster chains for directories, but the good news is that to play a file, you get its starting cluster from the directory (somehow... I'll get to that), turn the cluster number into the sector number (LBA is much easier than CHS, if you didn't already know), and then just read sectors and play them, one after the next as you decrement a counter for the file size (which also comes from the directory).

    Now, in the interest of keeping things simple, so you can get something working before you student project deadline is up, I'd suggest you use a pretty large cluster size, and make all the filenames fit short msdos names (8.3 sucks, but you can improve things after your initial success). If you use short names, each file will only consume 32 bytes in the directory, so a 16k cluster size will be able to hold about 512 filenames.

    For for your very first FAT32 attempt, you'll read the volume ID to learn a few critical numbers, which includes the first cluster of the root directory and a couple things you'll need to turn cluster numbers into sector numbers. The official FAT32 specification from Microsoft [microsoft.com] tells you which bytes mean what and what formulas to use. As you read it, their frustration with lots of poorly-written FAT code shows though...

    So, you'll grad the MBA (first sector, LBA=0), read the partition table to find out where the FAT32 volume starts, and then read its volume ID, and your two goals are to learn the first cluster number where the root directory starts, and the info you need to be able to turn that cluster number (and others) into the LBA number so you can fetch them. When you use the formulas from Microsoft's FAT32 spec, don't forget to add the offset due to the start of the partition, as their formulas don't include that (as I found out the hard way).

    Now if you've used short filenames, big clusters, and only a couple hundred files, then the entire root directory will fit into that one cluster! You read the directory to find the filenames, and for each one you get the two critical numbers you need, the first cluster and the file size. To play each one, turn the cluster into LBA, and divide the size by 512 (round up) for a count of how many sectors to play, and just start reading the sectors in order and send them to your decoder until you've done them all. Repeat for each file.

    Now that does have quite a number of limitations, but it completely avoids having to read any FAT sectors. With a fully defragmented drive and the entire root directory in just one cluster, you should be able to very easily get something working where a standard PC can write the drive (without the ugly perl script).

    My current code (0.5.1, available for free download, including GPL'd source) does follow cluster chains for directories, but not for files. It is pretty simple, though there is the added complexity of reading the FAT sectors. As I mentioned earlier, I'm very close to releasing a near-complete rewrite of the project, which will use the FAT for both files and directories, and it includes a malloc/free that's used by the FAT32 code to cache both FAT sectors and clusters. When you're ready to try something more complex, this free code may help you (assuming you can live with the GPL), but initially you should keep the project simple.

    Also, early in my project I wrote this page with detailed how-to instructions for interfacing an 8051 with a IDE drive [pjrc.com]. I recently wrote this how-to page about using the STA013 MP3 decoder chip [pjrc.com], which may help if you're planning to use the STA013.

    I hope some of my on-line material and code helps, and I hope your project turns out to be a success.

  • FAT is your best choice because of its simplicity. Porting ReiserFS (or even ext2) to a microcontroller would be a most majestically-sized project.

    Journalling isn't a requirement if songs are put on the player by attaching the disk to a PC -- when in the player, the disc is only ever read, not written.

    Just reading CDs doesn't make it any easier -- you have to read ISO9660. (I assume you don't mean audio CDs.)

    Peter

  • I don't know what you are using for your MP3 decode, but if you wanted a really easy solution try using the Cirrus Logic EP9312 ARM9 [cirrus.com]. It has built in EIDE, USB, LCD interface logic, kepad scanner, etc and can run embedded Linux. Of course this might be to easy of a solution for you (almost everything on one chip).

    That's about all i have to add.

  • Would it not be possible to write each MP3 to a CD as a mode 2(?) track, similar to the way in which the Video CD (White Book) standard is implemented?
    The player would then only need to read a raw bitstream off the disk starting at a given offset (track) position.
    Forgive me if this is totally ridiculous, but I've been playing around with White Book stuff for a while, and this seems possible (plausible? dumb? plain stoopid?).

  • I'd go with FAT. It's a lowest common denominator -- Windoze, MacOS, Linux, BeOS, heck, even GS/OS on the Apple IIgs can read it. It's also not as complex as you'd think, as long as you don't try to get too fancy with it.

    I'm wondering, however.. Given that an 66MHz 486 has issues playing MP3s, do you have enough horsepower with an 8 bit 30MHz processor to decode an MP3? It's 360 30MHz cycles per floating point multiply on the CPU.
  • True. But I can't see that an MP3 player could function with such a small drive.
  • I would recommend a journalling file system, because you don't want to lose your MP3 selection if you don't unmount your drive properly (which could happen otherwise). You are effectively restricted to an open source file system since it's not practical to run full Windows (implied by FAT) on such a device. You will be able to customize the kernel as well. Choose between ReiserFS and ext3fs.

    Also, although you don't say it's portable, if it is, a normal hard drive won't be suitable since they won't stand the movement. If it is portable, use a solid-state device.

    Finally, as to hard drive vs. cd, cd is much more flexible, since you imply that the device is a music player in function and hence there won't be any advantage from being able to download music onto the hard drive, and music players aren't much good unless you can change the album.
  • As you note yourself, most players at the moment seem to use flash memory or similar, not disk based. The exceptions I can think of just now are the 6Gb one that thinkgeek advertises on as a slash banner which must be disk based, and there are a couple of CDR based ones, but IIRC they have pretty bad production quality.
  • Aren't most portable players less than 100megs though? I mean my one is only 64megs of flash - some are only 32.

    I would want a larger drive, too.

    -- Eat your greens or I'll hit you!

  • I think the unmentioned thing here is the size of the drive.

    Despite ReiserFS's see-saw FS (ho!) it's allocation table overhead probably makes it not a good choice for small (less than 100meg) drives.

    OK, by now!

    -- Eat your greens or I'll hit you!

  • I was looking at some of the other posts, suggesting FAT (which is DOS-based, NOT windows-based), FAT32, etc... I don't think you even want to mess with that beast. Talk about bad coding.

    Why not just use Nero or something to burn your songs into an ISO image, then RAWRITE that image to your harddrive? Then, shouldn't you be able to read the drive almost like it was a CD?
  • Y0u r34lly w0uldn't 3v3n h4v3 t0 wr1t3 34ch mp3 t0 1t's 0wn tr4ck. S1mply c0nc4nt3n4t3 th3m 1nt0 4 f1le 4nd tr34t th3 f1le 4s 4n 1S0 w0uld b3.

    0f c0urs3, th3n y0u'd n33d t0 b3 5chm4ncy p4nts 4nd t4ck 4 l1st 0f pr3-sc4nn3d f1l3 0ffs3ts 1nt0 th3 b3g1nn1ng 0f th3 1m4g3. H4v1ng n0 tr4ck t4bl3 m34ns y0u h4v3 t0 h4v3 4n 4lt3rn4t3 w4y 0f d3t3rm1n1ng 0ffs3t. I supp0s3 0n3 c0uld d0 th3 0ffs3t s34rch f0r th4 0p3n1ng mp3 h34d3r, but n0t 3v3ry v4l1d str34m h45 0n3 4nd th4t w0uld b3 r4th3r t1m3 1nt3ns1v3 0n st4rtup.

    But, y0u C0ULD u53 th4t t1me t0 4ls0 c4ch3 th3 ID3 t4g5 1n 4 l0c4l db, 4l0ng w1th v4r10us 0th3r 1nf0rm4t10n 4b0ut th3 s3l3ct10ns 4nd 4 d1sc ch3cksum s0 y0u n3v3r n33d r3sc4n. It pr0b4bly w0uldn't b3 n0t1c1bly sl0w3r th4n 4 sc4n f0r th3 s4m3 1nf0rm4t10n w1th kn0wn f1l3 0ffs3ts.

    1 d1gr3ss. 'r4w m0d3' 1s unsu1t4bl3 f0r th3 t4sk b3c4us3 1t r3qu1r3s t00 much s3tup, s0ftwar3z w1nd0z3 skr1p7 k1dd133's l1k3 m3 d0n't h4v3, 4nd th3r3 1s l1ttl3 t0 n0 t3chn1c4l/f34tur3 m3r1t 0v3r a c0nv3nt10n4l IS0966o f1l3syst3m..

    3v1l_b0r1s at d4rkr0ck d0t c0 d0t uk

Slashdot Top Deals

PURGE COMPLETE.

Close