The Perception of 'Random' on the iPod

Robaato writes "Stephen Levy writes in the Guardian about the perception of randomness, or the lack thereof, on an iPod set to shuffle." From the article: "My first iPod loved Steely Dan. So do I. But not as much as my iPod did.... I didn't keep track of every song that played every time I shuffled my tunes, but after a while I would keep a sharp ear out for what I came to call the LTBSD (Length of Time Before Steely Dan) Factor. The LTBSD Factor was always perplexingly short." My first iPod shuffle refused to let me delete (sigh) Weird Al's Polkamon off of the flash memory.

And Zonk dupes himself... again...

[Anonymous Coward]

Is the iPod Shuffle Playing Favorites? [slashdot.org]

How about an analysis of the randomness of Zonk dupes. I guess I should be happy it's not a games story.

Re:And Zonk dupes himself... again...

[strider44]

I think that the slashdot editors can be forgiven for posting a link to an article on a similar topic a year and a half later...

That article is btw referenced in this one.

Re:

[MobileTatsu-NJG]

"And Zonk dupes himself... again..."

To be fair, the other story was feb of 05. Dupes suck, but do you really expect Zonk to remember every single story he's posted?

Besides, the article that was linked to was recent.

Re:

[GarfBond]

Not to mention it looks like Steven Levy essentially duped his own article for the Guardian, with the added benefit of time and history with the shuffle to make a conclusion.

Dupe Tag

[sr180]

And the question remains, why doesnt the DUPE tag work anymore? I liked that tag. Seeing it meant I could avoid the 500 "OMG! Its a Dupe!11!" comments.

Re:

[mikesd81]

Well, there is times I wanted to post an article but wasn't sure if it was posted yet and I searched for it. Surely he can too?

Re:

[MobileTatsu-NJG]

"I remembered it, and I didn't get paid to post it."

So? Do you remember every comment you've posted?

Re:

[tehshen]

So? Do you remember every comment you've posted?

I can't write a list of all of them, but if you give me a few, I can probably pick out the ones I remember.

Re:

[mrchaotica]

It might as well be; he's certainly not doing any editing!

Re:

[Somatic]

Even better, here's a quote from that thread:

Hey, shut up! It's a new post and it's NOT a dupe. Let's not push our luck, OK?

Bias

[Anonymous Coward]

http://en.wikipedia.org/wiki/Confirmation_bias [wikipedia.org]

Re:Bias - hmm

[sreekotay]

Not sure its so much confirmation bias (alone at least), as it is that the odds of NOT playing a song from the same artists over the next X songs shrinks more rapidly than intuition suggests. That is, for example the odds of NOT having a run of X heads or Y tails when flipping Z coins is very, very small.

The article mentions the "how many people does it take to get to a shared birthday thing" - and the point there is that its not that it takes 40 people to get to one with a SPECIFIC birthday but only 40 o

Re:

[IntelEmployee]

Blah Blah Blah........ Clusters are a common occurrence in randomness. If it played the same song every time you would have one extreme. If it played all the songs evenly a part, you would have the other extreme. Try something in the middle? That's where you will have clusters of a few songs played a lot, and clusters of some songs not played often or even at all. Want me draw you a picture? i iii iiiiii iii i It's all pseudo random anywho, and I don't think your clock cycle has a thing for steely

Re:Bias

[pchan-]

This is exactly what is going on. Good call, AC.

Some years ago, I worked on an mp3 playing device (no, not Apple). Our users were quite often complaining that our random was not truly random, and seems to be clustering, favoring, disliking some thing or another. Some would swear that there was some intelligence to it, picking particular songs. I've seen the shuffle code, it's a simple array swap. I ran a numerical simulation on the output and found that the distribution of the array elements from their original position equal throughout. Further, there seemed to be no specific clustering, as the probability that any item would end up next to any of its peers was again equally distributed throughout. We had some of the customers submit their own ideas and tried them out in code. In general, we found that we never outperformed the simple array swap in terms of randomness, though most results were about the same.

The conclusion that we reached: If you have a lot of Jimmy Buffet, you're going to hear a lot of Jimmy Buffet. And on that one occassion that two Buffet songs play back to back, you're going to think to yourself "this random sucks". But it is, in fact, all in your head.

*I'm sure someone will want to bring up the seed issue. Let's just say that we had it covered.

Re:

[Stooshie]

Just to expand on what pchan said, the chance of getting all the songs ordered by artist, is just the same as getting any other particular combination of songs.

Re:

[CreatureComfort]

This is exactly the thing I try to point out to people who play the lottery and insist that their numbers are far more likely to come up than the 1 2 3 4 5 I suggest they play.

Re:

[NeMon'ess]

Never-the-less, letting power-users control the randomness would be a nice feature. Say I have ten songs from an artist. When I make a playlist of a thousand songs, and the aforementioned ten all get played within the first three-hundred, that's not desirable to me. That means in the last 700 songs I'm not going to hear that artist, and that could mean weeks or months of playtime.

Apple does include an option for the minimum number of songs before playing an artist again, but that doesn't necessarily fix

Re:

[mgblst]

OK, that is fine. But that is not Random. You want something that is not Random, and that maybe the problem here. People have funny definitions of random, and random does not mean 1 of your 10 songs played every 100 songs.

Re:

[NeMon'ess]

Right. I want a controllable Shuffled mode to mix up the order. I don't want true Random play. I think that's what a lot of people want. Perhaps even a majority.

I don't even necessarily want 1 of those 10 songs played every 100. Last year I made a multi-thousand-song playlist in iTunes. After shuffling it and listening mostly through, I was noticing when tracks by Wolfstone played. I went back and realized that about 80% of those tracks had played in the first half of the playlist. Because the last

Re:

[GregWebb]

Agreed. Evenly weighted (or biased according to preferences) shuffle is far more useful than random.

In my last job, I listened to MP3s all day while working. I'd got RoboDJ feeding in the playlist, and Audioscrobbler logging the results.

Which turned out - that, remarkably consistently, Iron Maiden, Placebo and System Of A Down (typically 'Aerials' from Toxicity) got played more than their weightings justified, while Deep Purple measurably less than theirs. Including compensating for number of tracks held.

It

Re:

[NeMon'ess]

Almost two years ago. [slashdot.org]

Then they changed their form so they no longer accept feature requests for iTunes, only iPods. As for my request, iTunes 6 doesn't remember where I was in a playlist after closing the program, does version 7?

It only took Apple three or four years to incrementally improve their Shuffle feature. I'm sure I just need to wait another year or two for my request to get implemented.

Maybe in another two or three years enough people will have asked Steve Jobs to get the Shuffle feature to play

Re:

[GeffDE]

And I want iTunes to make me a continental breakfast every day by 7 sharp and then clean my house and prepare dinner for when I arrive home.

iTunes is a jukebox program, not your own robotic DJ. iTunes does give the option to weight the play-order based on how long it's been since a song played. Go to your library, turn off "shuffle mode," sort the library by Date Played and play the songs you heard last. In order. There you go. 100% weighting on date played. iTunes doesn't make a note in its databa

Re:

[Filiks]

Are you saying all playback programs shouldn't be sophisticated, complete, and comprehensive? Is Photoshop a bad program for being feature-packed? Or does it get a pass for doing content-creation, instead of content-playback? ACDC is an excellent, comprehensive program for reviewing and sorting pictures. Sports Illustrated uses it. Would you say ACDC needs to pull some of those features out because they make it too usable?

Now iTunes is obviously targeting non-power-users, but it still has some more adv

Re:

[Yer Mom]

Finally, iTunes doesn't make a note in its database if I've skipped a song before it finished or early on.

iTunes 7 does.

Mind, it doesn't do anything particularly useful with it, but you have Last Skipped and Skip Count available for Smart Playlists, so you could probably get some way towards what you want...

Re:

[Mattsson]

The problem with most players is probably not their random-generators, but the fact that they ranomize every song-change, not keeping track of what has already been played.
This leads to the probability of hearing one or more songs several times before all the songs in the player has been played.
What needs doing is ranomizing a playlist and then keeping to that until all songs has been played, then randomizing a new one.
Tadaa! No song is played twice in one pass! =)

And, yes, there are players that do this.

Re:

[RemovableBait]

Actually, this appears to be what iTunes does. Certainly for playlists.

Try it out yourself: load up iTunes, go to a playlist and hit the 'Shuffle' button. See how the playlist randomly reorganises? If you keep hitting the button, it toggles between normal, sequential order and a different, randomly-generated, shuffle order. Guess what happens when you finish listening to the playlist (and you've got it on repeat)? Yep, iTunes reshuffles and starts again. The only chance of a song being repeated is if the so

Re:

[geoffspear]

They don't. The iPod will play every song in your playlist (or your entire library, for that matter) exactly once if you leave it in shuffle mode until all songs have played. It will even display "Song X of Y" to let you know how far through the playlist it is, and reset the count when it's played everything (if you've got Repeat mode on). Of course, when the iPod is reset (or, more annoyingly, synced, at least on my 3rd generation one), it will forget where it was and play everything over again.

Re:

[Metteyya]

iPod shuffle (which the article is about) does not have hard disk, it has solid-state memory.

Re:

[Epistax]

I don't think confirmation bias is it. IDNPTIF (I do not prescribe to iPod fanboyism), but my iRiver's shuffle seems to have a problem. It may or may not be nicely random, but the issue is that it's the same random. Granted, it's several years old now, but if I load songs on there, by the third or forth cycle through on random, I can tell which song leads into what. I don't know about others, but on a CD once I play it through a couple times, I just know at the end of a song which one comes next, and my

OCD

[eldavojohn]

Or - and here is the nub of an issue that would consume me for more than a year - was the shuffle function, meant to mix up my music collection in a random fashion, actually not random at all?

There there, Mr. Levy, we'll get you all the randomness you want [wikipedia.org]. In fact, we have a special place filled with randomness and padded white walls! You're going to like it there.

You know, instead of wasting your interviewee's time, you could have installed a five song list on your iPod and set it to shuffle. You'd have to carefully mark down the track number being played and listen to it for 100 songs. Do this a few times and make sure you're very methodical about what you do. Wipe the iPod, put five songs on it in order and then listen to a hundred songs "randomly." If you start to see a pattern developing or one song is obviously favored over the other, it will begin to show up.

But on the more technical side, they have to seed the random variable with something. Whether or not it's an internal clock, I'm not sure. Either way, they have to derive a random number and it's possible that their seed isn't good enough or has too few states or is prone to being seeded at the same state, etc. Based on this information, I hate to break it to you but it is very hard to be truly random.

Re:

[jlarocco]

But on the more technical side, they have to seed the random variable with something. Whether or not it's an internal clock, I'm not sure. Either way, they have to derive a random number and it's possible that their seed isn't good enough or has too few states or is prone to being seeded at the same state, etc. Based on this information, I hate to break it to you but it is very hard to be truly random.

Just to be pedantic, they have to derive psuedo-random numbers. And although it is hard, there are doz

Re:

[ehrichweiss]

Holy crap!! "It was designed to have a colossal period of 2^19937 1 (the creators of the algorithm proved this property). In practice, there is little reason to use larger ones, as most applications do not require 2^19937 unique combinations."

To put that in decimal: 4.3154247973881626480552355163379e+6001. That's quite a number. I think that dwarfs "astronomical".

Re:

[timeOday]

As if any of this were relevant to music listening.

The simple truth is that the shuffle was an extremely lame product that was only created so Apple could cover the entire price range of mp3 players. Nobody else had the gall to sell a player with no display. "An experience in aural spontaneity..." pardon me while I barf. It was a simple matter of designing to a price. I won't question Apple on it because they've made more money from the iPod than I ever would have imagined. The folks who bought a Shu

Re:

[DrEasy]

Yes, there was a lot of marketing bullshit with the launch of the Shuffle, but it wasn't a bad product at all. All it needs is a way to support multiple playlists, for example with a "skip playlist" mode or button, so that you can navigate more easily between songs. There should be no need for a display to achieve this. It is a good thing to be able to navigate songs without taking the player out of your pocket.

speaking as someone who doesn't have any iPod

[leehwtsohg]

Doesn't the shuffle have the best sound quality of the set? (look it up through google)

Re:OCD

[monoqlith]

Not OCD. Sub-clinical schizophrenia [wikipedia.org]. . Read all the way to the end of the article(I'm new here). If you don't want to, I'll summarize: It's more about randomness than the iPod. He eventually realizes his suspicions of programmer malfeasance are in fact an expression of his own favoritism, not the iPod's. In other words, its all in his head. So a worthwhile, interesting article, and even if he could have benefitted by experimenting himself, that wouldn't have made for a very fun read, or an interesting question to ask Steve Jobs while he had the chance.

Whoa...

[PresidentEnder]

But on the more technical side, they have to seed the random variable with something. Whether or not it's an internal clock, I'm not sure. Either way, they have to derive a random number and it's possible that their seed isn't good enough or has too few states or is prone to being seeded at the same state, etc. Based on this information, I hate to break it to you but it is very hard to be truly random.

There must be bias on the internet. I remember reading the same thing not ten minutes ago in TFA... cre

Re:

[cp.tar]

There must be bias on the internet.

You mean... no net neutrality?

Sorry buddy, i disagree

[TheCouchPotatoFamine]

I think (and have... thunk?) that "randomness" on the ipod is actually a secret R&D weapon in the apple ipod toolkit. From a psychological standpoint alone, what is the value of all other mp3 players being truly (read unadjusted psuedo random) and the ipod being a little less.. that is, what if they, say, mark the number of times you don't let a song play through, but skip it in the first 10 seconds? There are powerful means by which they can onboard build a profile and i have three things to say about

Re:

[Khuffie]

From my brief experience with the iPod and iTunes player, their random features were never truly 'random'. For example, I liked David Gray's Ain't No Love so I'd usually start off with that and the iPod would be set to random. The next song would be, say, Coldplay, and the one after that would be Joshua Radin, and so on. So far, so good. Next day, I like to start with "David Gray's Ain't No Love", and sure enough, the next song would be Coldplay, followed by Joshua Radin, and so on. Until I add another song

SLAYER!!!!!!!1

[litewoheat]

My iPod likes Slayer and Marilyn Manson. I guess its posessed.

Mine loves Chevelle

[Private.Tucker]

...well, used to. Then I made a different playlist and labeled it as "Upbeat" music. Now It loves Motion City Soundtrack. Now, I like all the music I have on my iPod (duh) but its very noticeable when I hear the same song 3 times in one hour 30 minutes worth of driving. I can tell you that over the last 2 days (4 hours of driving) I have heard Foo Fighters' "Enough Space" 6 times out of 231 songs. Does the iPod sense higher played songs/albums/groups or is its randomness just that awful? 2GB Nano 1g

iPod metadata

[Meph_the_Balrog]

Does the iPod sense higher played songs/albums/groups

Actually it does. There's a counter for the number of times a song has been played through completely. I believe one of the in-built playlists accesses this metadata.

Mind you, as to wether the device uses this information to weight its shuffle function is something I have no idea about.

Re:

[Firehed]

I'm inclined to think so, as I seemed to get highly-played songs more often during random playback, almost as a pseudo-rating. Interestingly, it only seemed to happen on playlists, not during the "shuffle songs" entire-library approach (though, that could be because my library is at least ten times the size of my playlists, and I didn't leave it going that often).

If you do a shuffle of a playlist, album, or library, it WILL do each song one time per set, provided you leave it playing uninterrupted for that

Re:

[philipgar]

if the iPod/iTunes sensed higher played songs and played them more, it would actually cause a huge disaster, depending on how it was implemented. A simple implementation would increase the likelihood that a song gets played again by a fixed percentage. Under a situation like this, no matter how small of a percentage the song increases it's likelihood of being played by, and regardless of how many songs you have on the device, over enough time it will reach the point where it will effectively only play one

Re:

[mgblst]

...depending on how it was implemented.
 
Yeah, I think this is really important, don't you. I hope that if they implement something like this, they actually think about it, for at least as long as you have, what do you think?

Re:

[merreborn]

Some MP3 players just use a retarded algorithm. I had an MP3 CD player in which I stuck a CD with two folders: one with hundreds of songs, and the other with a single song. It played the single song half the time.

That's when I realized that when in shuffle mode, the CD player first picked a random directory, and then picked a random song from in that directory -- which happens to really, really suck in the edge case I'd presented it with.

Re:

[zlogic]

A truly random distribution can give you one song more than others. After all, it's random, right? So basically everything is possible, including the stuff people usually complain about. Now, if it carefully selects songs so that the same one won't play too frequently, that's not random.
You get uniform distribution only at really large numbers, at least 10^3 or more for a two-song playlist.

Old News.

[mh101]

I recall The Steve personally addressing this issue in one of his keynotes (although not with iPods, but iTunes). People thought iTunes' shuffle feature wasn't random enough. Steve assured everyone that it indeed was completely random, but then announced that iTunes had a new "Smart Shuffle" option. The description in iTunes is "Smart shuffle allows you to control how likely you are to hear multiple songs in a row by the same artist or from the same album." There's a slider with "More Likely" on one end, "Less likely" on the other end, and "Random" right in the middle. Although this feature is in iTunes, it has not yet made it onto iPods.

I personally have had it happen where my iPod is in shuffle mode and I've heard not just two songs in a row by the same artist, but a song plays and then the next song from that album follows it. And that's with a library of over 5,000 songs. Naturally it's more likely to happen on a much smaller Shuffle with a fraction of the songs.

Re:

[noidentity]

How funny, the original shuffle was random, but that wasn't considered random because people have this odd notion that random means that certain combinations will never occur. Now they have the new, improved random that's less random so that it'll seem more random. That's progress!

Re:

[Neoncow]

In one of my statistics classes, we performed a lab to emphasize the importance of random sampling. We were split into groups and given a set of 30 or so blocks of various shapes and sizes. we were to choose 10 blocks that were representative of the entire set. After each of the group presented their picks, the TA had a laptop randomly pick a set for every group. We added up the results. The random set was closer that the human guided guesses.

Re:

[u38cg]

This is just a variant of the Birthday Theorem at work (the idea that once you get more than twenty-something people in a room, the chances are two will share a birthday). Every pair of tracks has a possibility that you will consider them to be related, because they are the same artist, or the same song, or whatever. Given how many pairs of songs you have in your playlist, it is fairly likely that you will play closely related songs (to you) quite frequently.

Re:

[Dr Reducto]

You can set iTunes to shuffle by Artist, Album, or Song.

The problem with older versions was this wasn't wall documented (the preference was hard to find), and was set to shuffle by album by default. I know because I noticed the same thing, but eventually found out what I was doing wrong (the option was something like "Group songs by album for shuffle" or something like that)

Re:

[cpeterso]

I absolutely concur. I have a Creative Zen Vision:M and its "shuffle" sucks. If I add songs to my playlist in batches (e.g. one album at a time), the Creative Zen "seems" to shuffle the new batch, but then just append it to the end of the existing, shuffled list.

Truly Random

[sriramv_iyer]

I agree that it is extremly difficlut to be truly random. There are some good ways of initializing the seed in such a way that the pseudo random number generator behaves differently. A good way, done in telecom terminals is to measure the noise at the receiver and then use it to seed the random number generator. Since, the noise is truly random, that is a good way to seed the random number generator. If the costs, are not too high, then it might even be a good idea to read noise (or any truly random parameter) whenever required. That would be close to really random, provided, we can map the random parameter into a quantitative parameter without big errors and approximations.

radio static

[Bananatree3]

another really easy method is to build in a tiny, one-chip radio receiver inside the mp3-player and tune it to a very little-used frequency. The antenna only has to be long enough to get some good white noise. Radio static is more than random enough to seed a mp3 player.

Re:

[Jerf]

It's not cost-effective to be "truly random" with an iPod Shuffle's hardware budget, but who cares. It's not at all hard to convince a pattern-seeking human that it's not "random", because our pattern-seeking human thinks "random" means "play everything once, then start over with a fresh list", which is anything but random.

I just ran a simulation here of 100 songs, randomly chosen until all 100 songs had been selected once, and ran it 1000 times. On average, it took 523 choices to exhaust the list due to re

People are Pattern seekers

[BigDiz]

Humans innately seek patterns in things that are random. That's why so many people wear smelly socks because they think they're lucky. Once you identify a supposed "pattern" i.e. non-randomness, you're going to keep noticing instances that fit that pattern, and ignore instances that do not. This is deeply ingrained.

Think about it, if you're at the roulette table and black has come up four times in a row, how likely are you to bet black? Most people would bet red, because, I mean hey, there's got to be a pattern. But (as I'm sure you all can understand) black has the same probability of occurring again as red does.

People have had this complaint about all sorts of playlist randomizers (not just iPod), it's just people seeing what isn't there.

Re:

[Anonymous Coward]

Exactly, if you show people two series of dots, one really random and one evenly distributed, but not regular and ask them which is more random, they will say the evenly distributed one, becasue the random one has what we see as obvious patterns in it. So, I think what people want is evenly distributed (but not regular) mixing of songs, not true randomness.

Re:

[Erik K. Veland]

Good thing you posted that, it's not like that wasn't the entire point of the article or anything.

Two random modes

[daeg]

Apple should add another random play mode -- one that acts as it does now, and the other mode that grants every song an equal play count. The only thing that would be random is which order. This way users that have a confirmation bias of their iPod favoring certain songs can no longer be paranoid of Apple conspiracies to promote the songs of {{ artist }} or {{ record_label }}.

RTFA

[SanityInAnarchy]

This is just about exactly what "Smart Shuffle" does. It allows you to bias the randomness towards not playing two songs from the same artist or album in a row. Makes it less random to make it feel more random.

Re:

[Filiks]

Does it though? This guy [slashdot.org] just found that to play all 100 songs once took an average of 523 total plays. By that time the most-played song had played 11.3 times.

So if all Smart Shuffle does is prevent two songs from the same artist or album from playing too closely, it still might not even out the play count. It might mean that some songs still play three times in a list while some songs only play once. It's not good enough that over time they can average out, because users likely want consistently more-

SmartShuffle

[RzUpAnmsCwrds]

The open-source music player I wrote (BSoftPlayer) has a feature called "SmartShuffle". One of the biggest problems with shuffle is that it's difficult to understand when the tracks will change order, and it's difficult to know wheter or not a track is going to be played more than once in a single "cycle". Some shuffle features will play the same track twice before playing through your entire library, and some won't.

With SmartShuffle, the order is randomized, but it remains the same until you "reshuffle".

Re:

[SanityInAnarchy]

iTunes/iPod already does this, and it is, indeed, what "shuffle" means -- so long as you pause (instead of stopping or turning it off), you're still playing through a randomized playlist. The problem is spookiness of seeing five songs by the same artist in a row, or very close together, in that random playlist, but that's statistically likely anyway.

You might consider calling it something else. Apple is calling their new shuffle feature "SmartShuffle", but in this case, it's about creating a bias against

Re:

[mibus]

With SmartShuffle, the order is randomized, but it remains the same until you "reshuffle".

Is it different from the way XMMS et al. create their shuffle lists?

Winamp

[Ahnteis]

I do the same thing with winamp.

One of the playlist window options is "randomize list". I just hit that a few times, and then listen through the playlist "straight". Even better is that I can go backward through the same list, and can see what "random" song is coming up next.

It's because of the birthday paradox

[neomage86]

You can get the technical details here: http://en.wikipedia.org/wiki/Birthday_paradox/ [wikipedia.org]


The basic gist is that their are far more possible pairs than we'd intuitively imagine. For example, with 20 albums of 20 songs each, the chance of two songs in a row being from the same album is actually:
400/400 * 20/400 = 1/20
Which makes a lot of sense once you sit down and think about it, but is a lot higher than an uneducated guess.

This is the same reason that collision/timing attacks are feasible.

Re:

[mh101]

There are 12 people on staff where I work. 4 of us have birthdays in a span of 5 days, i.e. there's my birthday, one day off, then three birthdays in a row. I wonder what the odds of this happening are.

The downside of all this is we get gypped out of 3 birthday cakes this way since we only have one for all four of us. :(

Re:

[taustin]

Whatever the odds are, they aren't so high that it isn't inevitable that somehwere, there is an office with 12 people, five of whom have birthdays next to each other. It has to happen somewhere; you just happen to be it.

Re:It's because of the birthday paradox

[nine-times]

I think it's more than that. Yes, coincidences will happen, but I also think part of the issue with people perceiving patterns is that they can switch patterns whenever a new pattern seems to emerge. So, with reference to the Birthday Paradox, it's true that, in a party, it's more likely than you think that two people will have the same birthday, but what if you aren't bound by birthdays? What if you're just constantly looking for anything two people could have in common? If you're at the party constantly talking about dates, birthdays, anniversaries, favorite colors, food alergies, etc.-- then there's an excellent chance that you'll find there are lots of people in the party that something in common.

In the case of the iPod, i have an iPod and put it on shuffle often enough. For a little while, i'd always be suspicious that there was something going on. It seemed to happen way too often that I'd get two songs together off the same album or the same band, or I'd get a bunch of '80s songs together, or a lot of songs that I'd grouped in the same genre. You know, no specific pattern I could use to predict what would come next, but on any given day, I seemed to be able to find a pattern.

It wasn't always very conscious or thought out, but I'd catch myself thinking, "weird, I've heard 4 songs from the same album in the last hour. The iPod must not be mixing it up enough." But then I noticed some of my patterns were like, "huh, I've heard a couple Nirvana songs and Foo Fighter songs. My iPod must like Dave Grohl today." And then I realized, I didn't have the name "Dave Grohl" in any metadata anywhere. In order for the pattern to be caused by the library, you'd have to assume that the iPod's circutry somehow knew that Grohl was in both of those bands, but without any such link existing in my iTunes library.

So of course I got rid of the iPod, because it was obviously possessed by the devil and obessesed with Dave Grohl. I guess this guy [slashdot.org] is right.

Re:

[Rufus211]

Nitpick, the probability is: 1 * 19/399 ~= 1/20. The simple explenation is you pick any song, and then there are 19 songs from the same CD out of the remaining 399 songs.

RTFA

[mh101]

The article actually goes into more depth than people seem to be getting from the summary. The author is not complaining about problems with the shuffle. Rather, he starts by making note of his early observations, then describing his research into music shuffling, and how we perceive patterns where there are none.

FTFA:

First of all, note what it doesn't do - it's not like mixing all the songs in the equivalent of a big bucket of lottery balls and picking out the next one. Instead, as the name implies, it shuffles the entire library so as to reorder them, just as a blackjack dealer shuffles a deck of cards. If you listen to the entire library all through, you will hear every song once and once only. What is important, then, is not whether a song is included but how evenly an artist's songs are distributed throughout the list. When I say that Steely Dan is over-represented, it means that the band's songs show up early in the run - it would be like a blackjack dealer whose first hand had aces in it.

Or

We perceive trends when there are none. Poker players invariably believe they can lock into streaks. Backgammon champions swear that dice can go hot or cold. Likewise, people think they can cosmically predict what song will come next on their shuffle. The blogger Kapgar, who claimed this power, remembers vividly the times when he predicted a song and the iPod amazingly delivered it. But there may have been a thousand times when his iPod played songs he didn't guess - non-memorable circumstances that, not surprisingly, didn't make an impression.

What you should expect...

[YGingras]

A long time ago I was dissatisfied by the lack of random in XMMS so I jumped to the source to see what I could do. I think this was my first contribution to a free software project. Anyway, here is what I found: XMMS keeps two copies of the playlist, one that is in the order you set and one that is "shuffled". This has to be clear, all the tracks in the play list are there exactly once in the shuffled playlist.

I can't recall when the shuffled playlist was reshuffled but in was not that often, maybe only when you added or removed tracks. So if you like Smoke on Water but that Ballroom Blitz is just two song after that, too bad, you'll always get Ballroom Blitz soon after you double click on Smoke on Water. Technically speaking, the shuffling was perfect, the random generator was properly seeded and they divided in the right way to prevent loosing entropy. The lack of reshuffling was entirely responsible to the perceived lack of randomness.

So my patch was just that: trigger reshuffling a lot more often. As far as I know this patch was never merged but my copy of XMMS did have the proper random behavior. I don't know if it's the same problem with the iPod. That's something I like with free software: you can fix it!

xmms experiment

[jesterzog]

As far as I know this patch was never merged but my copy of XMMS did have the proper random behavior.

For what it's worth, I've just experimented with XMMS (1.2.10) in random mode, and it seems to be doing this now.

If I double-click a track to play it, then click advance, it was always advancing to a different track, implying that it reshuffled at the point of selecting a track. Simply moving backwards and forwards between tracks left them in a consistent order, however.

Re:

[YGingras]

Bug 21 in the XMMS Bugzilla [xmms.org], it was fixed in November 2002 by Håvard Kvålen. Not my patch but the general idea was kept. Ain't it a nice world we live in?

Re:

[NeoSkandranon]

iTunes seems to have the same behavior, and as far as i can tell, only reshuffles once a session. That becomes pretty obvious in a short playlist, when a song's likely to be repeated, or when you go "oh hey i want to listen to that Krokus tune again" and the same song always comes after it.

Reshuffling whenever you manually select a song seems like a good solution to me.

Netgear MP101 randomness

[M0b1u5]

I wonder if the "random" on an iPod is like the "random" on my Netgear MP101 wireless audio device connected to my stereo system.

If I select any playlist, and hit "PLAY" when the shuffle option is set to "Random" - then it plays the same order of songs every single time. To get a genuinely "more random" feel to the way it plays songs, I have to select the non-first song at the start of the playlist, then hit PLAY - and then hit "NEXT". After *that* it seems to be relatively random (Except for a pre-disposit

Check the play count

[Lumpish Scholar]

I didn't keep track of every song that played every time I shuffled my tunes....

Yeah, you did; or rather, iTunes kept track for you, as the "play count". Take a look.

(I looked at mine; it was closer to uniform than I'd perceived. There's also a "Skip Count", but it's blank for all my songs.)

Re:

[Phroggy]

Skip Count is new in iTunes 7, which is why it's blank for songs you've only skipped prior to upgrading.

Well, my iPod has GPS

[bedouin]

Because it seems the same songs play at certain points that I drive through in the city.

Or maybe it's not a grand conspiracy at all, but this grand idea called chance.

Human brains hardwired for temporal connections.

[Vellmont]

Is it possible that human brains just have the gain set a bit higher than simply "random" on connecting temporal causal events? It might be more adventageous to notice connections between events that aren't connected (then dissmiss them) than it is to ignore events that are connected causally.

In other words, it might be better to be a little over paranoid and think that the random shuffle on an Ipod isn't random, that childhood vaccinations cause (insert disease here) than it is to miss the fact that when

How about iPod Tetris' randomness?

[mh101]

Music shuffling is fine. What needs to be fixed is the randomness of the blocks in the iPod version of Tetris. Without fail, it always starts me out with the red "Z" block, yellow square, red "Z" and the brown "L" block for the first game upon starting Tetris (after this the next game starts with different blocks, unless I quit and reload Tetris).

Seems to me like the programmers used a bad choice of a seed value for the random number generator.

Uniform randomness

[FortranDragon]

I noticed that most people view a uniform distribution as a "random" distribution. As the article states, people impose patterns on things, thus when the average person tries to makes something random they tend to create something with a uniform distribution 'because it spreads the stuff out'. Ah, the fun of counter-intuitive math. :D

Similar to radio stations

[dodongo]

I know there will be snarky +5 Funny comments underneath this (as well there should be), but this system to decrease the perceived randomness is actually really similar to the algorithm most radio stations use when programming their music.

There's a simple parameter that's set to control, to within one minute, the amount of temporal separation there must be between playing two songs from the same artist, or the same song twice. The radio algorithm is a little more complicated, since songs aren't in just one big batch like the iTunes library, but in different categories, based generally on the perceived desire of target listeners to hear a given new song, or like and identify with a given older song.

The system is built off the (once literal, now metaphorical) use of index cards: The format clocks say, e.g., at the top of the hour, play a P category song, followed by a B category song, then a G, then an A, etc. You'd have a set of rules, like "don't play the same artist within 45 minutes" or "don't play the same current song within 3 hours", and you'd take the first card in the category that fit all the rules, play it, and move the card to the back of the stack.

Basically, what Apple is doing with that slider is enabling artist separation control, which is completely one of the illusions radio stations (used to) use to convince you they had every song under the sun available to them.

Re:

[feyhunde]

What is this thing called Radio? Is it like podcasting?

Soul of iPods

[webword]

Maybe iPods are showing us their souls. The inner light is shining through.

Humans and dictionaries define random differently

[jonadab]

Except for mathematicians and programmers, most think of "random" in a *very* different way from its technical definition. To most humans, saying that a particular sequence is "random" means *guaranteeing* certain things about it. Among them: the same element does not occur back-to-back, EVER, even if there are only a few elements total to choose from. Even more, if there are more than about half a dozen elements, the same element never occurs twice within about five positions. (So if you've got songs 1 through 7 on your iPod, and the first seven played are 5, 3, 7, 2, 4, 1, 6, then the next one has to be 5 or 3, or _maybe_ 7, or it doesn't seem "random" to most people. Yet, the order can't be the same every time through, either.) No element occurs substantially more often than any other element, even over the short term. If the elements have a natural order (e.g., alphabetical), then no three elements that are adjascent in that order can ever occur together in that order, nor should they occur together in the reverse order. (This gets particularly difficult to guarantee when the elements have more than one natural order, e.g., if the elements are people, you can't have three of them in a row by either name or age, or people notice and decide that the order is not random.) Even worse, if the elements can all be categorized into a small number of categories (e.g., by gender), you can't have "too many" from one category in a row. (How many is too many depends on the ratio, but if half of the elements are male and half female, having four of either in a row will make people cry foul, the order is not "random".) If certain elements stand out from the others in some significant way, they can neither occur first nor last. (For instance, if test questions are being drawn from a question bank, neither the easiest nor the hardest question should be first or last; if it is, people will say the order was not random.)

I could go on and on, but what it really amounts to is that when most people say "random" they mean "carefully arranged in a thoroughly mixed-up order". This is almost the *opposite* of what a mathematician or computer programmer thinks the word "random" means.

For this reason, when describing a mathematically-random sequence to an end user, I never EVER use the word "random". I generally call it something like "arbitrary" or "unpredictable". This greatly reduces complaints.

Now, as far as song frequency, I like to rate my tracks on a scale of 1-10, and rig my playlist so that anything under a 6 never plays unless I specifically select it, tracks rated 7 play twice as often as those rated 6, and the frequency keeps going up the higher my rating is. (I only have eight tracks rated as a 10, and they're all things I don't mind hearing back-to-back.) Then if I find a track is playing more often than I like, I figure I rated it too high and cut back its rating.

Re:Humans and dictionaries define random different

[doom]

Except for mathematicians and programmers, most think of "random" in a *very* different way from its technical definition. To most humans, saying that a particular sequence is "random" means *guaranteeing* certain things about it. Among them: the same element does not occur back-to-back, EVER, even if there are only a few elements total to choose from.

Yes exactly. I think the lesson here is that you should never use a mathematically random algorithm for esthetic purposes. If you're trying to get som

Problem solved...

[NotQuiteReal]

You only think you like Steely Dan, because others have told you likewise.

It is analgous to the popularity of Windows - there is no objective reason to like Steely Dan.

But, deep down, you realize, that Steely Dan represents the worst of the music of the 70's and you are having a visceral repulsion to it, like I finally came to realize, every time I hear Stevie Nix...

So, rather than blame your nice, techie, iPod, blame your fallable human self for choosing that music.

If you remove all Steely Dan from

No escape from tracks I hate

[British]

My ipod has this tendency to play tracks from the "Buckaroo banzai /saturn 3" soundtrack. It was a bootleg-only soundtrack that has about 8 BB tracks, and about 20 Saturn 3 tracks. I regret ripping it to my ipod. No matter if I use "shuffle songs" or any custom smart playlists, they ALWAYS show up.

And removing tracks from an ipod was never easy.

My old iRiver...

[MooUK]

I have an iRiver H320. The stock firmware for that player doesn't do shuffling properly at all. Every time you play something shuffled, it's in the same order.

The Rockbox firmware solves that, of course.

"The generation of random numbers is...

[ergean]

too important to be left to chance."

Like Scientoloigists extinguishing streetlights!

[Ancient_Hacker]

It's a psychological thing. Like walking by a streetlight and having it go on or off. If it happens again within an hour or so, you're SURE it had something to do with you. Even if it's the same streetlight. Even if you know when the bulbs get old, they overheat and cycle on and off. Every week or so I hear two Bob Dylan tunes back to back on my iPod. Not too surprising, I only have about 60 tunes on there and Senor Zimmerman "sings" four of them.

Intersting

[PurdueGraphicsMan]

Recently while mowing my grass I noticed that my iPod seemed to have a certain affinity for Sting. However, since I have just about every Sting album ever released I figure it's bound to show up a bit more often than some other artists. What I thought was odd was that it played like 4 Sting songs right in a row. The possibility of that happening seems remote considering I have around 3,000 songs on my iPod right now.

Re:

[Reaperducer]

I have one but only ever use it on long journeys and no I don't have DRM'd tracks so I didn't care about online music purchases. The ipod just happened to be the one that worked the best (scrollwheel is nice and quick) and having a mac I knew it'd work well.

It's a shame how people on Slashdot aren't allowed to just like iPods -- they always feel pressured to justify the purchase.

"Best tool for the job" isn't good enough. You have to be different. But only in a pro-Linux anti-iPod sort of way. Any o

Truly, a Slashdot legend

[punkass]

What, we're losing Anonymous Coward? He's been here since the begining, and he wrote half the posts! OH NOES!!!!1!

Re:

[SanityInAnarchy]

No, it's not! You have "al" four times in that, at least! Do you have something against Al! OMG CONSPIRACY!!!

The point is that true randomness doesn't look random, because randomness necessarily includes the possibility of patterns.

Re:

[Bert690]

Something related to iTunes - a study of the randomness of party shuffle in iTunes. This article does a bit of research and comes up with a function! Yep, and there was even a previous slashdot story [slashdot.org] about this very article. The function from the article to describe play probabilities grossly overfits the data. A much simpler formula is more likely correct, as I demonstrated in a comment [slashdot.org] submitted to this previous story.

Re:

[necro2607]

Ok, that's not really random though. That's just shuffling everything around and actually applying some logistical algorithms on the music - somethign like "mix up the songs so no two songs by the same band play in succession, and don't play any song more than once".