http://www.webmproject.org

It's what we were all hoping for, and I'm pleased that it's come to pass. It's not perfect, and I don't agree with all of their decisions. However, even an imperfect alternative to the current H.264 situation is a massive improvement. I'm pleased they've gone for a full stack solution of not only VP8 but also Matroska(-ish) and Vorbis, too. If nothing else, it's likely to mean that a) Vorbis will be shipped by default on most platforms[1], and b) hardware support for both VP8 and Vorbis should be widespread in the very near future. Further, there's a commitment to transcode all of the existing videos on YouTube. That's a massive endorsement. Of course, the risk with that is the reduced quality that will come from three or more lossy transforms, but given that they've announced it, they're clearly not too concerned about that. The future's looking very bright indeed.

[1] Or will it? I've yet to see any official response from Microsoft or Apple on this. But it's going to be hard for them to ship something that won't play YouTube videos by default. Of course, in the short and probably medium term, YouTube will continue to offer videos in other formats as well. But we'll see how it plays out in the long term.

I went to an interesting talk by Eng Lim Goh of SGI on Thursday, demonstrating their Ultraviolet systems. Very cool. While there are bigger and faster computers on the planet, they're all technically clusters rather than a single computer. The SGI machines can have 4096 CPUs and 16TB of RAM running under a single Linux kernel. Nice.

Also interesting to see the stats on when their systems were delivered to McLaren and Brawn, and where the performance of those cars was afterwards. I know there's more to F1 than CFD, but it certainly plays a large part these days.

The UK's Information Commissioner has ruled that research data must be made public. I have little sympathy for the likes of Keenan. But I have even less sympathy for so called scientists who refuse to make their data available for others to study. This isn't just about climate change, either, but about science in general. The recent spate of falsified results in Chinese and Korean papers should be ample evidence that data needs to be made public so that results can be independently verified. That's what science is all about.

My backup failed last night, because the device was read only:

building file list ... done
rsync: delete_file: unlink "/backup/local/shared/camera/2009-08-24--foq/img_0507.jpg" failed: Read-only file system (30)
rsync: delete_file: unlink "/backup/local/shared/camera/2009-08-24--foq/.xvpics/img_0507.jpg" failed: Read-only file system (30)

That's never a good sign. Looking through the logs, I see a number of ATA errors, starting with a timeout, a device error, and a bunch of HSM violations. A few examples:

Mar 22 07:56:03 riva kernel: ata2.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen
Mar 22 07:56:03 riva kernel: ata2.00: cmd 25/00:00:87:d1:86/00:02:03:00:00/e0 tag 0 dma 262144 in
Mar 22 07:56:03 riva kernel: res 40/00:00:00:00:00/ff:ff:ff:ff:ff/00 Emask 0x4 (timeout)
Mar 22 07:56:03 riva kernel: ata2.00: status: { DRDY }
Mar 22 07:56:03 riva kernel: ata2: hard resetting link
Mar 22 07:56:03 riva kernel: ata2: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
Mar 22 07:56:03 riva kernel: ata2.00: configured for UDMA/33
Mar 22 07:56:03 riva kernel: ata2: EH complete
Mar 22 07:56:03 riva kernel: SCSI device sdb: 1465149168 512-byte hdwr sectors (750156 MB)
Mar 22 07:56:03 riva kernel: sdb: Write Protect is off
Mar 22 07:56:03 riva kernel: SCSI device sdb: drive cache: write back

Mar 22 07:59:05 riva kernel: ata2.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x0
Mar 22 07:59:05 riva kernel: ata2.00: BMDMA2 stat 0x6d0009
Mar 22 07:59:05 riva kernel: ata2.00: cmd 25/00:00:df:ce:87/00:02:03:00:00/e0 tag 0 dma 262144 in
Mar 22 07:59:05 riva kernel: res 51/04:80:5f:cf:87/00:01:03:00:00/e0 Emask 0x1 (device error)
Mar 22 07:59:05 riva kernel: ata2.00: status: { DRDY ERR }
Mar 22 07:59:05 riva kernel: ata2.00: error: { ABRT }
Mar 22 07:59:05 riva kernel: ata2.00: configured for UDMA/33
Mar 22 07:59:05 riva kernel: ata2: EH complete
Mar 22 07:59:05 riva kernel: SCSI device sdb: 1465149168 512-byte hdwr sectors (750156 MB)
Mar 22 07:59:05 riva kernel: sdb: Write Protect is off
Mar 22 07:59:05 riva kernel: SCSI device sdb: drive cache: write back

Mar 22 08:01:15 riva kernel: ata2.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen
Mar 22 08:01:15 riva kernel: ata2.00: cmd 25/00:00:ff:1e:88/00:02:03:00:00/e0 tag 0 dma 262144 in
Mar 22 08:01:15 riva kernel: res ff/ff:ff:ff:ff:ff/ff:ff:ff:ff:ff/ff Emask 0x2 (HSM violation)
Mar 22 08:01:15 riva kernel: ata2.00: status: { Busy }
Mar 22 08:01:15 riva kernel: ata2.00: error: { ICRC UNC IDNF ABRT }
Mar 22 08:01:15 riva kernel: ata2: hard resetting link
Mar 22 08:01:15 riva kernel: ata2: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
Mar 22 08:01:15 riva kernel: ata2.00: configured for UDMA/33
Mar 22 08:01:15 riva kernel: ata2: EH complete
Mar 22 08:01:15 riva kernel: SCSI device sdb: 1465149168 512-byte hdwr sectors (750156 MB)
Mar 22 08:01:15 riva kernel: sdb: Write Protect is off
Mar 22 08:01:15 riva kernel: SCSI device sdb: drive cache: write back

SMART doesn't seem to be much help here, with the device refusing to run anything but the mandatory offline test:

riva:~# smartctl --test=offline /dev/sdb
smartctl version 5.36 [i686-redhat-linux-gnu] Copyright (C) 2002-6 Bruce Allen
Home page is http://smartmontools.sourceforge.net/

Default Self Test Successful
riva:~# smartctl --test=short /dev/sdb
smartctl version 5.36 [i686-redhat-linux-gnu] Copyright (C) 2002-6 Bruce Allen
Home page is http://smartmontools.sourceforge.net/

Short offline self test failed [unsupported field in scsi command]
riva:~# smartctl --test=long /dev/sdb
smartctl version 5.36 [i686-redhat-linux-gnu] Copyright (C) 2002-6 Bruce Allen
Home page is http://smartmontools.sourceforge.net/

Long (extended) offline self test failed [unsupported field in scsi command]

It's a Seagate drive:

Host: scsi1 Channel: 00 Id: 00 Lun: 00
Vendor: ATA Model: ST3750528AS Rev: CC38
Type: Direct-Access ANSI SCSI revision: 05

I'm running 2.6.18-128.4.1.el5 on CentOS 5.

Any ideas? Does anyone know enough about the ATA spec to tell me what these errors actually mean? Is this a genuine hardware failure? If so, where's it likely to be? Drive? Controller? Cable? The drive is only a few weeks old, so while sometimes shit happens, I want to investigate the (probably more likely) alternatives as well.

How can I get a <td> to be a containing block? Section 10.1 of the spec implies that just giving it position: relative; should be sufficient. But that's completely bypassed, and the search carries on up the element hierarchy. I kludged it by putting a relatively positioned <div> inside the cell, but it's frustrating to have to resort to such workarounds. Anyone know why it didn't work? There's no mention in the spec of table elements being given special treatment when it comes to containing blocks. Indeed, it explicitly mentions that table cells should work. But they don't (in Firefox 3 and Opera 10, at least).

And why oh why doesn't <tbody> nest? I want to refer to part of a table and manipulate it with some Javascript (mostly just to show or hide it in response to user actions). But I can't just stick the relevant bits in a <tbody> and select by that because the table already contains several tbodies. Thus I'm once again stuck in workaround hell, trying to tag each tbody with an appropriate class, so I can iterate over them and check if I'm interested in each one. Further, because I'm dynamically adding and removing parts of the table, I need to check at the time I add to the table whether the bit I'm adding should be visible or not. Life would be so much easier if I could just stick it in an appropriate <tbody> further up the hierarchy, and then mark the whole lot visible or otherwise. But I guess that would be far too easy :-(

The time has come for a new laptop. But how do I find the right one? My requirements are minimal. Namely, I need a large screen, decent battery life and a low price. Surely there must be a laptop comparison site somewhere that lets me enter my specs and it'll give some recommendations, right? Yes, I can go to Toshiba's site and there's a great tool that does exactly that. Dell's and Acer's are adequate but not quite as good as Toshiba, and HP don't have one at all. But all of them are restricted to a single brand (as you'd expect). Where is the cross brand equivalent? Further, there are no doubt brands I've never heard of that I might be interested in, and without knowing who they are, I can't go to their web sites to check their offerings even if I wanted to.

Any CPU will be more than sufficient for my needs. The screen needs to be at least 17", and not gloss (which sadly seems to rule out the otherwise promising models from Samsung). I need 2GB RAM, and I don't want to be paying more than £500. A bit of digging reveals that all of the major players have models that meet those requirements. But I still have the nagging feeling that I'm missing out of the "right" model simply because I don't know about it. Where should I be looking?

Oh, and it won't have an Apple badge, so don't even consider suggesting that...

I don't know why, but I always get a warm feeling when I configure policy routing. Under Linux, iproute2 makes it almost trivially easy, so there isn't any real reason for it, but when I see my packets going off where I tell them to, I always seem to have a sense of accomplishment that's disproportionate to the amount of difficulty involved.

We had a power cut this morning. The second in two days (yesterday was at work, this one was at home). It was bigger than I thought, too, extending at least as far as the tube station, as I found out when I tried to go to work (trains weren't stopping due to the lack of power at the station). But it wasn't such a big deal. I went to work, reasonably confident that by the time I got home, the power would be restored. Sure enough, I returned home to find all the machines powered up. The main server was sitting waiting for a root password because the journal recovery had failed, and it needed an explicit fsck. Fair enough. But after fixing that, everything appeared to be back to normal. Except is seemed more than a little sluggish. You see, after the return of power, my ADSL router had decided to connect at a whopping 256Kb/s down[1]. Nor was it just a freak of the line state when the power returned. Disconnecting and reconnecting gives me exactly the same speed. Naturally, my utterly useless ISP is closed at times when it would actually be useful to call them. So I'll have to wait until tomorrow morning to get them to try testing the line, and generally failing to have a clue about anything, if past experiences with their technical support team are anything to go by :-(

[1] And 512Kb/s up, FWIW. The A in ADSL usually means you get much better bandwidth down than up. Not so here, it seems...

Today, I received an email. It's not quite spam in that I gave them my address when I went to a gambling conference a while back, but morally, it's spam. Normally I'd just delete it, but the conference is on again this week and is just down the road, so I gave it a quick skim thought. One of the products being advertised caught my eye:

Money Controls is using IGE 2010 in London this week to launch the ccTalk DES - Data Encryption Standard - encrypted products. The product is a response to customer demand for improved machine security in places where fraud is a constant menace, such as arcades, motorway service stations and pubs.

Uhhhh... that can't be right. It's 2010 and they appear to be talking about launching a product based on DES. Maybe they're using the phrase in the generic sense, rather than the US NIST encryption system of the same name. However, they go on to say:

A significant deterrent in combating fraud, the products use over 72,057,594,037,927,936 combinations.

7.2 x 10^16 combinations? That looks suspiciously like 2^56 to me. Yep. They really are launching a new product based around 56-bit DES, an encryption system that has been theoretically broken for 20 years (probably for longer than that by people keeping it quiet) and practically broken for at least 10 years. The same DES that was withdrawn as a US federal standard 5 years ago, even in its stronger Triple-DES form (which this product doesn't appear to be using). Yes, that DES. How are they not only not embarrassed about this, but actively boasting about it in product release blurb? Fine, so maybe encryption isn't their core competency. But surely even a cursory glance at the web when considering which encryption algorithm to use would be a good idea? At the very least, it would point you in the direction of AES/Rijndael or something (anything) other than DES.

So Verizon have apparently decided to block incoming mail from my primary SMTP server:

<<< 571 Email from xxx.xxx.xxx.xxx is currently blocked by Verizon Online's anti-spam system. The email sender or Email Service Provider may visit http://www.verizon.net/whitelist and request removal of the block.

Now the IP in question is mine. It's a static IP. It has never been used to send spam. My guess is that they've just randomly blocked a larger range that happens to include my /28. Ho hum. I suppose that just means people on Verizon will have to do without my words of wisdom. It serves them right for choosing to use a moronic ISP. On the other hand, it also means that my girlfriend can't get in touch with their customers if they happen to be on Verizon, which is more of a problem :-( I hate large ISPs who are so inept that their attempts to protect their customers results in splash damage like this. Sadly, it seems to be a feature, once they get beyond a certain size.

Yes, yes, and more yes. This is all too familiar. I've had the misfortune of having to use this abomination. Sadly, I'm going to have to do so again in the near future, too :-(

http://i.imgur.com/MpUyV.jpg

I have two tables. One represents individual customers. The other represents groups of customers. For the application I'm writing, I want to be able to apply restrictions to certain things, and I want the flexibility to be able to do that at either the customer level or the group level. So each table has a cutoff column, which says that any order with an ID lower than the cutoff should be ignored for a particular operation. So I can quite happily do:

select c.cutoff, g.cutoff from customers c, groups g where c.group_id = g.id and c.id = 45;

However, what I want to do is:

select mymax(c.cutoff, g.cutoff) from customers c, groups g where c.group_id = g.id and c.id = 45;

The problem is, mymax() doesn't seem to exist. I can't use the regular SQL max(), because that's an aggregate function that works across multiple rows. I want to return the larger of two columns in a single row. Oh, and it should know that any given integer is greater than null, because the common case is for there to be no cutoff, and only very rarely will both a customer and a group have a cutoff. I came up with a hideously complicated way of doing it using coalesce(), but it was so nasty I resorted to doing the comparison in python instead. Am I missing something? Surely this should be easy in SQL. I'm using PostgreSQL 8.3, FWIW.

I'm somewhat nervous right now. You see, my main server is moving datacentre. My hosting provider rents rack space, and then resells it to small operations like me. But the datacentre have just doubled their costs, so my hosting provider is taking their business elsewhere. I can't say I blame them. But it means my machine has to move. The problem is, that's something that's fraught with danger. Servers tend to run continuously. On the rare occasions when they're powered down, there's a reasonable likelihood that the drives will fail to work again. Empirically, the chance is far, far higher than for a regular desktop machine that is switched off every so often. The last time I did this, I had 6 drives fail in a 14 drive array. From personal experience, I'd say each drive has only an 80-85% chance of survival.

Of course, I'm a firm believer in not putting all of your eggs in one basket. So my machines are hosted in two separate datacentres, and although the main server is powered off (and I've decomissioned the other one that was in the same datacentre), I still have fully working DNS, SMTP and the critical web sites are all up. In the event that the main server has problems, I can move the non-critical web sites over, too. I have multiple copies of all the data. But still, it's an anxious time. I'll find out if all's OK in about 6 hours. Like I say, fingers crossed...

I have no phone. Its battery ballooned. Furthermore, the spare battery I had is refusing to charge, so I'm currently without a mobile phone. But the thing is, it's surprisingly liberating. I have long despised the telephone. It's an instrument of the devil. This is mostly because I'm not great at holding a conversation. I struggle to think of things to say. I'm really enjoying not being contactable at will. I feels as though a weight has been lifted from my shoulders (one that I didn't realise was there in the first place). Carphone Warehouse not only don't have a new battery for me, but they can't even order one[1]. I've ordered one from a mail order place on the net, and it will no doubt arrive in a few days. But even when it arrives, I'm considering using my phone less. I suspect I'll still take it with me when I'm out and about. But I may well have it turned off much of the time. I like the feeling of freedom.

[1] What's that about? It's not as if it's a prehistoric phone. Sometimes I despair at the increasingly disposable nature of the consumer electronics market.