I've been spending a lot of time away from my momma's basement* and have been mostly been hanging out at my special lady-friend's place.
She has a 2Mbps connection, and depending on who is visiting, there can be a half-dozen people actively using it. With her old router (a D-Link box that only supports stock firmware and DD-WRT), everyone hated the Internet here: It barely worked. There were nonsensical discussions about "how many people were using the Internet" when things would slow down.
With Shibby's version of Tomato USB, I set up some QoS rules on an old WRT54G. I gave her own laptop a slight preference, but really: With QoS, multiple independent Netflix streams are working OK, even though the boy streams Youtube almost continuously. The Sonos plays music from Spotify perfectly. Interacting with the Web is fast and responsive from all devices. Torrents don't bog the connection. Interactive ssh, RDP, and VNC are very usable.
Nobody complains now. Sure, downloads are slow, but downloads will always be slow here compared to at my own place in Mom's basement. Things just work, and the streaming stuff (except Sonos, which gets a high priority, because buffering audio is teh suck) scales to available bandwidth on a minute-by-minute basis, and it all seems to work fine.
Prioritize small streams of all types: Who cares what they are, they'll be gone almost instantly anyway and their impact is therefore small. These are things like NTP, DNS, the initial handshake of every HTTP connection, and other little stuff.
Prioritize important stuff: The HTPC is probably a better streaming target than some stranger's iPod Touch who you gave your WPA passphrase to just to be a nice guy.
Progressively penalize larger transfers: A lot of loading a modern Web page is the initial HTTP handshake, and a mile of CSS includes. Getting these done quickly is not so expensive in terms of absolute bandwidth availability, but really does improve the user experience.
Eventually, the classes go to a "bulk" category: Something over a few megabytes, or somesuch: If the game/windows/whatever update takes forever, so be it, as long as you can still use the rest of the internet while that transpires.
One can also do the "deep packet inspection" game, which is well-supported in Shibby, and gain a little bit more control. But that's decreasingly useful as more and more connections are encrypted by default (and one cannot inspect an SSL-ish packet without also performing a MITM attack upon the whole connection).
Rant: This is what IP TOS flags are for, but they're almost universally useless because end-user programs STILL do not (or cannot) use them properly. But if they were used properly, I could totally accommodate that low-latency VoIP or interactive SSH session, at almost-zero expense to Netflix streams.