Yes, yes, and more yes. This is all too familiar. I've had the misfortune of having to use this abomination. Sadly, I'm going to have to do so again in the near future, too :-(

http://i.imgur.com/MpUyV.jpg

I have two tables. One represents individual customers. The other represents groups of customers. For the application I'm writing, I want to be able to apply restrictions to certain things, and I want the flexibility to be able to do that at either the customer level or the group level. So each table has a cutoff column, which says that any order with an ID lower than the cutoff should be ignored for a particular operation. So I can quite happily do:

select c.cutoff, g.cutoff from customers c, groups g where c.group_id = g.id and c.id = 45;

However, what I want to do is:

select mymax(c.cutoff, g.cutoff) from customers c, groups g where c.group_id = g.id and c.id = 45;

The problem is, mymax() doesn't seem to exist. I can't use the regular SQL max(), because that's an aggregate function that works across multiple rows. I want to return the larger of two columns in a single row. Oh, and it should know that any given integer is greater than null, because the common case is for there to be no cutoff, and only very rarely will both a customer and a group have a cutoff. I came up with a hideously complicated way of doing it using coalesce(), but it was so nasty I resorted to doing the comparison in python instead. Am I missing something? Surely this should be easy in SQL. I'm using PostgreSQL 8.3, FWIW.

I'm somewhat nervous right now. You see, my main server is moving datacentre. My hosting provider rents rack space, and then resells it to small operations like me. But the datacentre have just doubled their costs, so my hosting provider is taking their business elsewhere. I can't say I blame them. But it means my machine has to move. The problem is, that's something that's fraught with danger. Servers tend to run continuously. On the rare occasions when they're powered down, there's a reasonable likelihood that the drives will fail to work again. Empirically, the chance is far, far higher than for a regular desktop machine that is switched off every so often. The last time I did this, I had 6 drives fail in a 14 drive array. From personal experience, I'd say each drive has only an 80-85% chance of survival.

Of course, I'm a firm believer in not putting all of your eggs in one basket. So my machines are hosted in two separate datacentres, and although the main server is powered off (and I've decomissioned the other one that was in the same datacentre), I still have fully working DNS, SMTP and the critical web sites are all up. In the event that the main server has problems, I can move the non-critical web sites over, too. I have multiple copies of all the data. But still, it's an anxious time. I'll find out if all's OK in about 6 hours. Like I say, fingers crossed...

I have no phone. Its battery ballooned. Furthermore, the spare battery I had is refusing to charge, so I'm currently without a mobile phone. But the thing is, it's surprisingly liberating. I have long despised the telephone. It's an instrument of the devil. This is mostly because I'm not great at holding a conversation. I struggle to think of things to say. I'm really enjoying not being contactable at will. I feels as though a weight has been lifted from my shoulders (one that I didn't realise was there in the first place). Carphone Warehouse not only don't have a new battery for me, but they can't even order one[1]. I've ordered one from a mail order place on the net, and it will no doubt arrive in a few days. But even when it arrives, I'm considering using my phone less. I suspect I'll still take it with me when I'm out and about. But I may well have it turned off much of the time. I like the feeling of freedom.

[1] What's that about? It's not as if it's a prehistoric phone. Sometimes I despair at the increasingly disposable nature of the consumer electronics market.

http://lwn.net/Articles/354860/

Sometimes you can really appreciate living in a world that isn't 100% driven by a company's bottom line. Can you imagine the uproar if Gates, Ballmer or Jobs had come out with a comment like that? Yaaay Linus.

Some while back, spam became a sufficiently large problem for me that I decided to do something about it. After looking around at the options, I decided to go with bogofilter. It's small, fast, and written in C. These are all good things. It looks at each message and builds up a database of words and the frequency with which they appear in spam and ham, and uses that to classify each incoming message. So far so good. It seems to work very well. When it's not sure, it says so, and files the message in an "unsure" folder, and asks me to manually classify those messages. It also uses that clarification to further learn the difference between ham and spam for future messages. This all sounds ideal.

The problem is that my main server was starting to spend a significant proportion of its time in disk wait state, while the bogofilter processes tried to access the word database. The load average hit 100 last night, which was an indication that things had gone badly wrong and I needed to do something about it. I don't know how Berkeley DB is implemented internally, but my guess would be that it's vague hash table shaped, and so the size of the database shouldn't significantly affect lookup times. But I don't know that for sure, and my database had grown to a non-trivial 1.5GB in size. Furthermore, bogoutil was failing to read the database file, possibly indicating that it was corrupt, which might have been causing each bogofilter pass to take much longer than necessary, and hence increase contention for the database file on disk.

So at this point, I decided to scrap the word database and start again. I have a corpus of both ham and spam that I used for the initial training, which I knew wouldn't be perfect, but it was a good start, and I could further train it up over time. Then I left it and went to bed. This morning, I realised just how much work bogofilter had been doing on my behalf. You see, in the 12 hours since I switched over to the new word database, I received 18,000 spam messages that bogofilter was now no longer filtering out for me. I knew the spam problem was bad, but that's at least an order of magnitude worse than I expected.

The next step is to train up bogofilter again so that it filters the majority of those out for me, and given the volume of mail that I'm getting, I think I'm going to need to use bogofilter to feed a Jef Poskanzer style dynamically updated block list that I can pass to blackmilter. 18,000 messages (plus however many bogofilter blocked) in 12 hours? That's insane.

Bizarre, I posted this earlier today, yet it seems to have disappeared. Maybe I just didn't hit submit after the preview. Anyway:

<?xml version=\"1.0\" encoding=\"UTF-8\"?>

Grrr. If you're going to stick that at the top of your feed, please mean it. 0xA0 is Latin1, not UTF-8. Lying to me like that just makes my life miserable :-(

I was looking forward to installing the recently released Fedora 11. But sadly, it was not to be. The installer fails. And by that, I mean it doesn't even get as far as the installer itself. I can't get beyond ISOLINUX. The "Press Enter to begin the installation" message appears to mean "Press Enter repeatedly and bugger all will happen". Sadly, I can't even report it as a bug. The Red Hat bugzilla has decided that my password has expired, but doesn't want to send me a password reset token :-(

In the last few days, I've been posting a number of replies to articles but apparently Slashdot has decided that they are not worth displaying because when I go back and reload the page they are not there anymore, even after giving it some time in case there is an automatic moderation overlord that might need some time to "approve" them...
Anyone care to comment or explain?

It would appear that censorship in Australia has married the anti-"fair use" campaign by the *RIAA...
I wonder if by reporting the issue that would also be considered indirect linking to the harmful content (well, supposed harmful...)
I also wonder how they can make sure that the intermediate links are still active and thus bring to the offending page without breaking the chain at some point - maybe they use some sort of link-checker for HTML websites? - otherwise it wouldn't be offending anymore, right?

The software giant is finally admitting its defeat: "Where do you want to go today?" seems more of an answer to themselves than a slogan for the community of paying customers...
(They already admitted defeat a while ago in the hardware department)
Here's a list of what "good" competition brings to the users:
OOXML Microsoft is adding full support for OpenDocument (ODF) file format to their Office suite of business productivity software with SP2, were not for the competition (already 10% of the market) that they are receiving from OpenOffice.org (now owned by Oracle).
Windows 7 for netbooks
Who would have thought that an OS that normally retails in the few hundred dollars would be able to be included in computers of roughly the same cost? (these days netbooks tend to be more on the $400-$500 range than the original $200 for the EEE PC 701, but anyway...)
Try before you buy
Vista was in beta for 1 year, and Windows 7 release candidate will also be free to use until 1st June 2010. You have to admit, it's a pretty big win for users (due to competition from Apple MacOS X and GNU/Linux), even though corporate ones always had it the easy way.

Where do you think the industry could be if in the last 20 years Microsoft would have not been allowed to persevere in its illegal activities? (remind yourself of the court proceedings before proclaiming them innocent...)

It would appear that there isn't right to privacy anymore on "the tubes": FBI spyware

Why on Earth a US court would have jurisdiction to authorise access on computers of people leaving abroad or using third-countries proxies is beyond me...

[original post from 19th April 2009]

Seeing the recent trends of:
- Firefox users quickly upgrading to the latest version
- Internet Explorer users stuck with company policies and the likes

I can easily predict the prompt jump of Firefox 3.5 to the top spot in all of the 5 continents (or 6 continents, depending on how many fingers there are in your hands) as the single most used browser version, followed by Internet Explorer 7 & 8 and a few monsters still using version 6. Do not also forget the growing share of Safari users, already over 5%, as iPhone and iPod Touch users account for 50% of mobile Web access...

Long live the king

(btw, that would be summer in the northern emisphere... those in latin America, Africa & Oceania will have to wait until Christmas)

[update on 9th September 2009]
The prediction is well underway, with Europe as the first continent where it happens: http://gs.statcounter.com/#browser_version-eu-daily-20090810-20090908

[update on 23rd November 2009]
The day that it happened: on 22nd November 2009, Firefox 3.5 was the most used browser version worldwide !!! http://gs.statcounter.com/#browser_version-ww-daily-20091105-20091122

The USA finally took the top spot in a category no one would want to be: the country where most Internet attacks are coming from: link