It wasn't a sensor bit error that it failed to guard against. The control values I referred to are those in RAM, used by the software. The RAM apparently wasn't parity protected, and a bit-flip in the right word could cause uncontrolled acceleration. It wasn't the only thing that could cause havoc; there were race conditions and stack overflows in the code, apparently, and those were more likely the sources of actual, observed UA.

This lengthy article at EE Times digs into some of the details. The main quote, though, is on page 3:

Memory corruption as little as one bit flip can cause a task to die. This can happen by hardware single-event upsets -- i.e., bit flip -- or via one of the many software bugs, such as buffer overflows and race conditions, we identified in the code.

There are tens of millions of combinations of untested task death, any of which could happen in any possible vehicle/software state. Too many to test them all. But vehicle tests we have done in 2005 and 2008 Camrys show that even just the death of Task X by itself can cause loss of throttle control by the driver -- even as combustion continues to power the engine. In a nutshell, the fail safes Toyota did install have gaps in them and are inadequate to detect all of the ways UA can occur via software.

I don't think that article pointed out this other detail:

Although the investigation focused almost entirely on software, there is at least one HW factor: Toyota claimed the 2005 Camry's main CPU had error detecting and correcting (EDAC) RAM. It didn't. EDAC, or at least parity RAM, is relatively easy and low-cost insurance for safety-critical systems.

This particular set of problems at Toyota was very interesting to us at work. I'm just now starting to work with our safety critical team that sells hardened controllers into the automotive market. They include all sorts of hardware failsafes, including ECC, lockstep execution between parallel cores, etc.

That may be true, but you do see cell towers and cellular basestations, which are similar in a lot of ways to data-centers. It's just that their data is phone calls and whatever data you're streaming over high-speed links.

I recall seeing Sun Microsystems had a facility in Denver when I drove through there around a decade ago. You'd think they noticed.

And yet the Slashdot summary makes it sound like something new. I know at work we always quote our error rates with a location and elevation (eg. New York, sea level), and I understand that's the standard way to do it.

This stuff comes up in deep embedded systems too. Think "ABS brake controller," etc. BTW, this is part of why Toyota got in so much trouble with its drive-by-wire system—it had no parity checking on critical control values. Granted, in an automobile, you have plenty of other sources of potential bit errors, such as extreme temperatures, power issues, exposure to strong fields, etc. But, you gotta protect against them all.

I remember a quote, attributed (likely incorrectly) to Seymour Cray: "Do you want it fast, or do you want it accurate?"

If you want absolutely exact arithmetic, code it entirely with arbitrary precision exact integer arithmetic. All rational real numbers can be expressed in terms of integers, and you can directly control the precision of approximation for irrational real numbers. Indeed, if your rational numbers get unwieldy, you can even control how they are approximated. And complex numbers, of course, are just pairs of real numbers in practice. (Especially if you stick to rectangular representations.) If you stick to exact, arbitrary precision integer arithmetic and representations derived from that arithmetic that you control, then you can build a bit-exact, reproducible mathematics environment. This is because integer arithmetic is exact, and you have full control of the representation built on top of that. Such an environment is very expensive, and not necessarily helpful. You can even relax the order of operations, if you can defer losses of precision. (For example, you can add a series of values in any order in integer arithmetic as long as you defer any truncation of the representation until after the summation.)

If you venture into floating point, IEEE-754 gives you a lot of guarantees. But, you need to specify the precision of each operation, the exact order of operations, and the rounding modes applied to each operation. And you need to check the compliance of the implementation, such as whether subnormals flush to zero (a subtle and easy to overlook non-conformance). Floating point arithmetic rounds at every step, due to its exponent + mantissa representation. So, order of operations matters. Vectorization and algebraic simplification both change the results of floating point computations. (Vectorization is less likely to if you can prove that all the computations are independent. Algebraic simplification, however, can really change the results of a series of adds and subtracts. It's less likely to largely affect a series of multiplies, although it can affect that too.)

And behind curtain number three is interval arithmetic. That one is especially interesting, because it keeps track at every step what the range of outcomes might be, based on the intervals associated with the inputs. For most calculations, this will just result in relatively accurate error bars. For calculations with sensitive dependence on initial conditions (ie. so-called "chaotic" computations), you stand a chance of discovering fairly early in the computation that the results are unstable.

Your argument only makes sense if you fix the target platform, compiler and compiler options for the comparison. In fact, it's trivially provably correct: If the compiler beats my assembly code, I can simply replace my own assembly code with the assembly code the compiler generated and force a tie.

However, that misses the point: I can write the fastest possible assembly for a given platform, but it might not be the fastest way to do something on a different (but compatible) platform. But the C code, without modification, could potentially beat my unmodified assembly code when compiled for that other platform. The compiler has the flexibility to tune its output for the target, while my assembly code is fixed for one target. And if you include platforms with a different underlying assembly language, the C code wins by default because my assembly code doesn't even run.

For example consider 4x4 matrix multiplication. Do you use nested loops or just unroll it manually? Compilers tend not to fully unroll all the nested loops. The compiler may do better scheduling on fully unrolled non-looping code.

Ironically, a vectorizing compiler would prefer you give it the loops instead. If you gave it any hints at all, it should be regarding pointer aliasing (ie. the C99 restrict keyword, for example), pointer alignment and minimum trip counts if any of the loops have variable trip counts. Manually unrolling makes its job much, much harder, usually.

Do you create temporary variables to preload a row or column, or do you just access each variable in memory directly? The former may generate better code on a RISC architecture and the later on a CISC architecture.

If you provide good pointer aliasing qualifiers, I'd hope both produce about the same regardless of CISC or RISC with modern compilers, instruction schedulers and register allocators.

These are the sort of things I think of when referring to helping the compiler, giving it hints. When that mythical smart compiler arrives that is able to figure out the preceding on its own, it will simply ignore your hints, the hints will do no harm. Until this mythical compiler arrives, the hints may help.

When was the last time you used the register keyword in C and it had a meaningful effect? Depending on which aspect of the code you consider, the "mythical" compiler you refer to may be less mythical than you think.

Look up the history of the Stepanov Benchmark as it applies to C++ programs, for example. It was once a hot topic among C++ compiler writers, because it exposed how awful C++ abstractions were to run time. Now most compilers ace it, sometimes producing faster code with the C++ abstractions than the C baseline they're measured against.

Ok, maybe not 20 years old, but 17 years old. Software I wrote in 1996 is still used today to verify chips built in the team I'm in at work. And that code compiles just fine. I haven't developed actively on it in about 14 years. No substantial tweaks to keep it current, either. I don't think it will compile as a 64-bit executable, but given that even Firefox is available as a 32-bit executable by default tells me that that's not a "historical" mode.

I was speaking with a team at work. They're talking about finally replacing some 30+ year old code in their code base with more modular, modern code. Sure, the whole package around it has continued to evolve, but some pieces date back to the first Reagan Administration. High level languages made that sort of continuity possible.

Now granted, the team whose code I'm referring to is a compiler team. Maybe, just maybe, they put more faith in compiled high level languages than your average programmer. ;-) But, their work on high level and low level optimization has paid off again and again. We have a kick ass compiler, and I say that as a hard-core assembly programmer whose job was to set the bar them. (I don't expect a compiler to beat me on assembly code I spent a month on. But, I expect a programmer using C/C++ with intrinsics and directives to get to 95% faster than I could in hand-coded assembly. That last 5% is what we call "diminishing returns," folks.)

LLVM started outside of Apple. Apple hired some key LLVM developers, and put several of their own on it too. They've kept it public because enough people outside Apple are still contributing, and sure, that's great. So far everyone benefits. If Apple decided to stop publishing their LLVM updates and took it private, FreeBSD would have to fork it or move to another compiler.

But none of that is specific to FreeBSD, and none of those fund core FreeBSD development (which could happen just as easily with GCC or another compiler if LLVM were unavailable). Your point, again?

I personally find it more valuable to periodically examine the compiler output, and understand if there are particular idioms in my code which lead to bad code generation. For example, when I'm constructing a particular set of abstractions (say, a C++ template library), can the compiler peel them back and give me efficient code?

I still write assembly code when I need to (especially on small embedded systems, or on the Intellivision), but most of the time I just don't have the time. Also, most code's performance just doesn't matter. My time truly is the limited resource. I've had to learn that perfect is the enemy of good, and so to pick my battles wisely.

MenuetOS is impressive, but I doubt I would ever be able to use it, because I won't ever have the subset of peripherals, motherboards, etc. required to run it. And, because its development is entirely in assembly code, I suspect hardware will continue to change faster than it does.

I remember the last time I bet on an assembly-written horse (WordPerfect). It was the fastest, it was solid, and it got trumped, hard, when the world changed faster than it could.

I'm not sure why you think x86 is at all opcode compatible with the 4004. It's not even opcode compatible with the 8085. (You could translate 8085 code to 8086 with a special translator, but it wasn't guaranteed to be perfect.) The 4004 had a very weird instruction set, actually. Probably had something to do with the fact that you had to manually manage the memory bus and chip selects from the CPU, as opposed to more generic memory busses found on, well, just about anything outside the microcontroller world that talks to a JEDEC memory.

Otherwise, most of the CPU complexity that currently shows up is due to the fact that the CPU speed far outstrips the memory bus speed, thus all of the concern about "local" memory caches and pipelined instruction ordering. If you could create a much faster memory bus, CPU designs could be simplified considerably from a software developer POV.

This is the infamous memory wall, and lets face it, no processor vendor has figured out how to bypass physics and "just make memory faster." The problem was identified as far back in the 1940s, long before Intel even existed or Shockley's team at Bell Labs had invented their transistor. Consider this quote from Von Neumann himself:

Ideally one would desire an indefinitely large memory capacity such that any particular ⦠word would be immediately available. ⦠We are ⦠forced to recognize the possibility of constructing a hierarchy of memories, each of which has greater capacity than the preceding but which is less quickly accessible.

A. W. Burks, H. H. Goldstine, and J. von Neumann
Preliminary Discussion of the Logical Design of an Electronic Computing Instrument (1946)

But hey, I'm sure you've got some ideas. Why not get some VC money and make the next processor that will beat them all?

As I recall, FreeBSD provided some of the key underpinnings to Mac OS X and iOS. Surely Apple can spare some of its $90B back to the effort. $1M is a rounding error compared to $90B...

Not trying to do the "one up" thing here, but IMHO, the end game for evolution would be to become beings of pure information. Energy and matter are merely vehicles to store and transfer information content. We would probably get equally frustrated with the limitations of existing as energy beings as we currently do with the flaccid biological bags that we exist in.

And your DRM comment is indeed something to ponder on - the artificial copy protection mechanisms that we have slapped on top of our existence - not just at physical levels but even in our minds.