I don't underestimate it, I use Google to find solutions to problems all the time. Google has more answers to problems with Oracle software than Oracle's official documents do. The point I was trying to make is that we wouldn't be spending money to hire consultants if Google had the answers to our questions. We never brought consultants in for any problem that we hadn't been trying to solve via an SR through our official support contract for at least two or three weeks and we were always well past the "have you checked Google" stage of problem solving, yet the consultants provided by Oracle (for outrageous fees) never had any useful contributions or problem solving approaches beyond "Google it". We were trying to hire knowledgeable professionals with experience troubleshooting problems with very specific pieces of a proprietary software component. What we got were living versions of lmgtfy.gom

I couldn't agree more. I've been working on an Oracle based JMS-SOA system for the past year. I've opened over 30 SR's in that time that have lead to over 20 bugs being filed. The development team has told us that they won't be able to fix some of the show-stopper bugs we've discovered until next December (as a year and a half from now). I have weekly meetings with Oracle product managers where they give me the same song and dance about how hard they are working to fix our issues despite never getting any closer to providing us with a functional product. We've spent millions of dollars on licensing fees and hundreds of thousands on consultants. Four out of five of the Oracle consultants we've hired have been completely useless. I'm talking useless to the point where they were just sitting next to me and searching Google for answers to the problem we brought them in to solve. Never hire Oracle consultants for anything more complicated than installing a database. We have ~15 very competent engineers on this team and we've finally gotten upper management's approval to begin a working on a proposal to move away from oracle products to open source or in-house solutions after six months of completely useless support and schedule slips caused by Oracle software not working as advertised.

The Papal States, Vatican City, and arguably The Holy Roman Empire

If you think DARPA is funding "development of multi-scale computational models with high spatial and temporal resolution that describe how neurons code declarative memories " because they care about veterans and not because they're looking for a more effective way to pull memories from people's minds than water-boarding, you haven't been paying attention to how America treats their military veterans.

The debtor's prisons that we've started running. Because we've reverted a medieval mentality.

You expect to be billed for it. With interest.

They stand farther away from the speakers than you do :)

I said "speaker wire" not "speaker". :) i agree $1k isnt THAT outrageous for a really nice set of speakers, but youre an idiot if youre spending that on speaker wire unless youre buying it by the mile.

People who describe themselves as audiophiles tend to be the kind of people who think that spending > $1000 for a speaker wire will improve the quality of their sound. They're really not worth paying attention to.

It's because a vaginae provide pretty reliable timers that let one predict when the crazy is going to arrive well in advance. Predictable crazy is usually more more annoying and tedious than it is exciting or interesting.

edit: I meant to type "from a steadily growing powerhouse into the shitty cycle of booms and busts"

Yeah, the neo-liberal set seem to have forgotten how the Reagan administration gutted SEC regulations and cut the tax brackets of America's richest by over 50%, which converted our economy into a steadily-growing powerhouse into the shitty cycle of booms and busts we're currently trapped in. Allowing banks to grow "too big to fail" and letting Wall Street create financial instruments so complicated that even their industry leaders can't explain to congress are indicators that de-regulation went WAY too far.

I mean how the costs don't outweight the benefits. Dammit, I always proof-read what i think I wrote, not what I actually wrote.

An uninitialized variable can be caught with a style-checker. There's no need to resort to something like randomized binaries to solve a problem like that. I'm not arguing in favor of leaving bugs in place, I'm arguing in favor of choosing a specific set of binaries to focus your testing efforts on. The bottom line is that testing resources are finite and one of the key steps to fixing a bug is identifying a method of repeatably demonstrating that bug. Having randomized binaries severely complicates that one critical task and will result in significantly lower quality testing when utilizing the same level of resources.

I agree with you completely about cross platform development being one of the best methods of exposing bugs, but i don't think this kind of stack randomization is really comparable. When doing cross-platform development, you'll have a very specific, very well-defined set of target environments that you'll be testing a single version of software on. This stack randomization is an effectively infinite number of variations on a theme being tested in a single environment. One lends itself to repeatable testing, the other lends itself to versioning hell trying to replicate bugs in order to solve them.

I agree it's worth looking into, but I'm currently having difficulty seeing how the costs outweigh the benefits.

You respectfully disagree with his points without actually providing any reason why, and while nick's post makes complete sense, your statements seem to have a ton of unexplained assumptions built in.

1. What kinds of bugs do you think would manifest earlier using this technique, and why do you think that earlier manifestation of that class of bugs will outweigh the tremendous burden of chasing down all the heisenbugs that only occur on some small percentage of randomized builds?
2. How does such an environment reward programmers who invest more time in validation? More time spent in validation will result in better code regardless of whether you're using a randomized or non-randomized build. More time spent in validation is a cost you're paying, not some free thing provided by the randomized build process.
3. I don't know what this sentence means: "Debugging suck, if instigated soon enough to matter, returns 100x ROI as compared to debugging code." If what instigated soon enough?
4. "Determinism should not be reduced to a crutch for failing to code correctly" - What does this even mean? An algorithm is either deterministic or non-deterministic. If your build system is changing a deterministic algorithm into a non-deterministic algorithm, your build system is broken. If your algorithm was non-deterministic to begin with, a randomized build is not going to make it any easier to track down why the algorithm is not behaving as desired.

All in all, your post reads like a smug "Code better, noob!" while completely ignoring the tremendous extra costs that are going to be necessary to properly test hundreds of thousands of randomized builds for consistency.