I saw recently that FreeBSD 8 was in BETA state. I ran 7-CURRENT for a while, because it had features I wanted to test (improvements to the OSS implementation mainly), so I thought I'd give it a try.
This time, rather than doing my usual source install, I tried a binary upgrade using freebsd-update. What a disaster. While the source upgrade procedure uses mergemaster to update configuration files, letting you just keep the new version of files you haven't modified, freebsd-update makes you merge them all by hand where there is a conflict. This wouldn't be a problem, except that all of the config files have a version line at the top, which conflicts between the two versions.
Inevitably, when manually handling the merge for a few dozen files, I missed an important bit so my first boot failed with an error complaining about the diff lines still being in the file. I fixed that, and rebooted.
My next boot failed because one of the startup scripts had replaced an if statement with a case. Unfortunately, this hadn't shown up as a conflict, so it had just taken the start of the case statement and the end of the if, giving nonsense. Fortunately, I was able to find the correct version in CVS and copy it out.
Next boot, my network interfaces weren't working. Actually, this was a problem I'd found earlier. When you update FreeBSD, you update the kernel, reboot, then update the userland (the new kernel is guaranteed to support the old userland, but the converse is not true). The em driver for Intel GigE cards complained that they both had invalid MAC addresses. Not a huge problem; it's a VM so I could just change the kind of virtual network card it was providing to the machine, but checking the bugs database I discovered that it's giving the same error for people with ThinkPads that actually do have this kind of hardware built in. Great.
Finally, my system decided to fail to boot with the error:
mounting /etc/fstab failed, startup aborted
Strange, I thought, I wonder which disk is failing to mount. A quick check in single-user mode showed that everything in fstab had mounted correctly. I eventually tracked this down to a bug in /etc/rc.d/mountcritlocal. This is not present in CVS, so it's probably introduced by the merge process. The value of $? (the exit value from the last command) is stored in $err, another command is run, and then there is supposed to be a switch statement branching on $err, which instead is branching on $?.
I've run betas, release candidates, and even the development branch of FreeBSD before, but 8-BETA2 is the first time I've ever had a FreeBSD install that feels like a beta. The merging done by freebsd-update seems completely broken; it prompted me for things it could have trivially done automatically, but failed to prompt me when it broke random system files. My system is now working again, but it's irritating to have to spend this much effort on an update.