I lost two friends and my father this year. I dedicated this release of cerowrt ( http://cero2.bufferbloat.net/cerowrt/credits.html ) to them. Most of the machines we have are named after someone that has passed, for example our main build box is named after http://en.wikipedia.org/wiki/John_Huchra It helped a lot to channel them all as we struggled to get the releases out. And, surprisingly, making ice cream, with liquid nitrogen as the coolant, has got to be a healing ritual, around here.

What Jim and the bufferbloat.net's group of volunteers have accomplished in a year - on nearly no money - boggles my mind.

Today's commentary on slashdot is a hundred times more clueful than it was last year - and a few days back Byte Queue Limits went into linux's net-next tree, which fixes much of the bloat problems that exist at the ethernet driver layer.

What has been discussed as 'Time in Queue' limits in the higher level schedulers is still awaiting a clean way to avoid layer violations. I've been too distracted by the BQL merge to pursue that next phase of fixes.

What we could have done this year with *some money* - nowhere near the amounts you describe above! - could have been amazing, and as for the next year, well, who knows? It is going to take many man-years worth of effort to make the internet responsive again.

And even with that said, to have harnessed the powers of hundreds first, now thousands, of talented minds, to help solve the bufferbloat problem - has been a far more effective - and wonderful! thing than all the money in the world.

Nearly every Linux machine ships with named (bind9) available and often, even turned on, in a caching-only configuration. To use it by default you just disable /etc/dhcp/dhclient's domain-name-servers request and point your resolv.conf to localhost. By doing this you get NXdomain back, too... and your local cache of dns entries is likely to be more performant than an ISPs 10s of ms away for cached entries. You can also run dnssec, if you so choose. Latest versions of bind can do dnssec, you can enable it with one line in the conf file. Ever since multiple services started messing with DNS a decade ago... returning broken queries, pointing to ad sites, not doing ipv6, not returning mx records, etc... I've run my own dns server. Now that dns is being mis-used for censorship, perhaps more will rebell. As servers go, in memory it's rather small...

on google maps http://arstechnica.com/tech-policy/news/2011/03/google-maps-300tb-of-real-world-internet-speeds.ars

The original gatech study showed not only bufferbloat, but enormous variation of base latencies in the first mile for different brands of cable modem as well as for different kinds of DSL and wireless technologies.

Slides: http://www.caida.org/workshops/isma/1102/slides/aims1102_ssundaresan.pdf

Some commentary: http://gettys.wordpress.com/2011/02/17/caida-workshop/

I look forward to the followup!

Sort of in answer to both of your questions the bufferbloat.net servers are configured as follows:

http://www.bufferbloat.net/projects/bloat/wiki/Dogfood_Principle

trying at every point to make sure http 1.1 actually got used.

We survived today's slashdotting. Handily.

That said, your points are well made. SPDY is part of the chromium browser and looks to have some potential.

In my case, I like the idea of smarter - and eventually sctp-enabled - proxies, especially on wireless hops. See thread at:

https://lists.bufferbloat.net/pipermail/bloat/2011-February/000068.html

Please feel free to write some code. Writing a new qdisc for Linux and BSD is not very hard.

Writing a good qdisc, insanely so.

That said, I tend to feel that time-stamping more packets and doing more guessing may make sense, as does concepts in TCP vegas.

But: Before starting, read these:

http://pollere.net/Pdfdocs/bcit_6.2001.pdf Kathleen Nichols - who proved to Van Jacobson that RED was wrong -

And Van Jacobson: http://pollere.net/Pdfdocs/QrantJul06.pdf

Which would you rather have? CPU time or personal time?

re:"Interesting problem for dd-wrt"

Agreed.
We are throwing efforts at both the mainline kernel and openwrt.
Openwrt is foundational for dd-wrt and several other (commercial) distributions of Linux on the router. I have a large set of debloated routers already, I'm just awaiting further work on the eBDP algorithm to make better....

http://www.bufferbloat.net/projects/bloat/wiki/Experiment_-_Bloated_LAGN_vs_debloated_WNDR5700

re: "using pings"
httpping is a much saner approach than ping, in many cases. Get it from:

http://www.vanheusden.com/httping/

re: RED & AQM

SFB and CHOKEe are in the debloat-testing kernel, as is eBDP.
RED 93 isn't going to work. nRED may. Experimentation and scripts highly desired. See the bloat and bloat-devel mailing lists for discussions.

Also:

http://www.bufferbloat.net/projects/bloat/wiki/Dogfood_Principle

Also:

I've seen some VERY interesting behavior with tcp vegas over bloated connections.

http://www.bufferbloat.net/projects/bloat/wiki/Experiment_-_TCP_cubic_vs_TCP_vegas

Well... yes, I'm one of the first 144 on bufferbloat.net But jg defined Bufferbloat so well, that the packet traces I'd seen on my wisp6 design in South America, suddenly made complete and total sense, when I first saw his back in November. I'd had no idea what I was dealing with was actually a worldwide probem. but thank you for the thanks. I blush.

I think you are referring to an early draft of an article by Eric Raymond, which was wildly incorrect on this point, which was thoroughly discussed on the bloat mailing list. See the incredibly long thread starting at: https://lists.bufferbloat.net/pipermail/bloat/2011-February/000050.html There is a newer, fully corrected piece coming out soon. In the meantime, consider the words of Van Jacobson, Vint Cerf, and others. Carefully. https://lists.bufferbloat.net/pipermail/bloat/2011-February/000108.html http://www.bufferbloat.net/projects/bloat/wiki/Quotes

you are conflating - to some extent - two things that a lot of people get mixed up. 1) on the TCP/ip sending and receiving side of the hosts, you already have very big, dynamic buffers in the stack for managing BDP. In this case, without very smart queue management, the TX_QUEUE and the DMA TX ring are completely "extra", and mess up the BDP calculation. There are no "extra buffers" in the TCP equations for the host side. 2) on switches and routers, large receive buffers are ok, for BURSTS, with queue management, flow control, and other AQM features. TX, not so much, and in the general case, small buffers are simply better as you need to signal end-to-end that there is indeed congestion, before catastrophy happens. There are other ways to lower the impact of flooding an individual host from a big fat server, for example multiple forms of weighted fair queueing.