So, like all the other old gits ... I got a Wii on launch day. Not really said much about it as I've spent too much time playing on it. If you want to XP my Mii's or whatever, friend code is: 2790-4876-4470-0181.

So you've probably all seen the 80 core CPUs in 5 years story.

One thing that I immediately thought (after, OMG WTF!) was "what the hell is the UI going to be like for that". For instance top currently has a summary line for each core, but having over 3 terminal screens (80x24 is as god intended) just for the summary seems unlikely to be appreciated. On the other hand having a program with a single task have CPU usage with an upper bound of 1.25% CPU seems less than useful. This seems really bad (Ie. can't treat that many cores seperately and you can't treat them as a single unit).

There's also the (probably huge) set of minor problems with things like And-httpd starting sysconf(_NPROCESSORS_ONLN) number of tasks by default. Which works well for the 1-4 CPU/core case, but is likely to be annoying with 80 cores.

Any thoughts?

So, I'm not sure if this is all tomhudsons fault ... but it seems like any random moron can get stuff posted to /. nowadays. Hmmm, maybe I'm just getting old and cranky.

Exibit A, so we're going to do a "DB benchmark" to see how well the different OSes compare to each other. Ok, I can buy that ... so you're using MySQL, which isn't that tastful, but I guess PostgreSQL isn't stupid enough to use threads and maybe that's what you want to test ... and people do use MySQL, so I can forgive that, I guess.

However, when running a "real world" test on different OSes it seems like crack to install an upstream MySQL tarball ... no sane person is going to be doing that, if they can get an errata supported MySQL package locally. Sure, there are going to be differences apparent from the different versions ... but that's going to be true if you deploy them. And if any of the OS vendors did any work on their package, that's now lost too.

And then to give a "methodology" that says:

I installed each operating system "stock" -- with default options -- and recompiled a kernel here and there when necessary, to add, for instance, SMP support in FreeBSD or increase memory allocation limits. For the most part, I didn't do any special OS tweaking beyond what was specifically recommended for MySQL. Any changes to the OS I made are documented in their respective sections below.

Oh, a recompile "here and there" and some random MySQL tweaking that doesn't appear to be documented and for the "non-special OS tweaking in Linux":

I ended up going with Gentoo 2004.3 ... With Gentoo it was also relatively easy to install NPTL for 2.6 ... (although they [ed: NPTL] didn't make any difference) ... For these tests, I chose ReiserFS version 3.

hahahahahahahahahahahahahahahahahahahaha. Ahhh, I have to laugh or I'll cry.

Well I released Vstr-1.0.13 last night, it's now "fully tested". I've posted some stats. etc. on my LJ "blog", or you can just install the example programs including a http server.

I'm pretty sure it's the most tested open source code, so one the one hand it's hard not to brag ... but on the other hand I can't help thinking to myself, "Well done, you've written a HTTP/1.0 server with quite a few options missing and it only took you about eight years :o).".

Still I'm running the example httpd on my ADSL, and I'm pretty sure it's secure :).

Is it me, or does noone actually read the web. One giant hypertext inverted WORM device.

Maybe, I'm getting cynical in my old age ... I mean I've seen random people write things that "I know" has been written about as being false for months or years. But I kind of expected IBM research ... to at least have done, you know, a little research before posting articles.

But hey, I guess Fox is allowed to say it's a "NEWS" channel ... so maybe I've just not been cynical enough.

So, as some of you may know, I'm pretty interested in scalable network IO. So I was pretty excited about the recent /. article. Alas, it was written by Felix "I write broken code" von Leitner of dietlibc and libowfat "fame".

Apart from the fact it mainly talked about dealing with network IO events, it completely screwed up the definition of the poll() API ... to the point where all the poll() related data is worthless (see socket_poll, which does what he says you can't with poll ... and faster). This includes the point about fork() being faster then poll() on Linux (which I'd put money on not being true). I was also sad to see alarm() even as a consideration for a timeout API, IMO do not use alarm() ... ever.

It only touched on iovec's and scatter gather IO from the application POV, which is probably due to zero support for that from the DJB designed buffer_* string IO APIs. Also the connect() calls were all blocking, and I think the data traveled using httpbench never used more than a single socket at once (Ie. there weren't ever multiple sockets seeing data at once). Sigh, I guess I'll have to finish my text for all that stuff ... and here I was hoping someone would do it all for me :).

On the other hand, there were a couple of interesting points I didn't know...

• epoll now does Level triggered events by default (stupid, stupid, stupid...). People should just learn to deal with edge triggering properly (of course I'm slightly biased as Vstr helps out a lot here).
• DJB has an IO API
• DJB wants to do non-blocking IO, I'd assumed DJB was firmly in the fork() and blocking IO model after qmail.
• Felix has written a read only LDAP server ... it's very possible that even being Felix this will be better to salvange than anything from OpenLDAP anytime soon.
• The description of the O(1) schedular was pretty good.
• The description of level triggering events and edge trigger events, was also pretty good.
• It's probably always good to have advice out there telling people not to use threads, even if I'm not sure the numbers are right.
• It is nice to have a simple reference for all the IO event mechanisms, this would be better somewhere on c10k really though, IMO. esp. as it's more likely to be accurate.
• I didn't realise you could make BSD TCP_NOPUSH actually do anything useful, although TCP_CORK is vastly nicer it appears you might be able to get generic code to use TCP_NOPUSH on *BSD with most of the gain.