Wall Street has attracted the best and the brightest of all of our people, math PhD's, ... Our most brilliant citizens are pulled into Wall Street as "quants" ... And to do what? To game the system in favor of their wealthy masters at the expense of the middle classes.
Then the PHBs misunderstand and misapply the PHDs' work, and the whole thing comes crashing down on them.
Case in point: Mortgage-backed securities.
Risk on such things is hard to estimate, because it takes a lot of investigation and skull-sweat to evaluate the risk on each mortgage. Evaluating the risk on a bundle of mortgages was so much work it was not practical.
Then the young math whiz proved that price of mortgages was very strongly correlated with risk, and came up with a formula that, given price, estimated risk very well. (Well, DUH! They're correlated because smart buyers and sellers were researching the mortgages, determining the risk, and basing their trading prices on them.)
THen the PHBs came up with something like bonds backed by a "basket of mortgages" (to "average out the risk of individual defaults). Buy the bonds (to finance the mortgages), get paid dividends from the borrowers' payments. Sell THREE sets of bonds against each "basket" of mortgages, with missed payments coming out of the dividends of the third, then the second, then the first, so investors could get different prices and risk/reward tradeoffs from the same basket. So far so good...
But to sell these bonds they needed a rating. So they talked the rating companies into using the shiny new risk-estimating tool to rate them. Oops! Any controls engineer who understands these bonds and the market will recognize that this substituted a positive feedback loop for the signal from the real world. Higher price -> lower risk estimate -> higher price... (The guy who did the original work said not to use it this way - but nobody listened. And he moved on to other things.)
And now that they could get a rating they could get a rating from reputable companies they could sell a bunch of these bonds. So they could buy up mortgages to make more. So this raised the demand for mortgages, which raised the price. The positive feedback loop was kicked off with a big up-push, the ratings went sky high, the prices of the bonds climbed, and the bubble was on.
With the price skyrocketing more people wanted to buy in. So the demand for mortgages went through the roof. Banks and the like could sell any mortgage they could write, even to "NINJA" borrowers with no income, job, or assets. Who cares if some of the loans in the basket are "subprime"? The price says the aggregate risk is low and it will all average out, right?
So the bubble blew up bigger and bigger, with developers building more houses that were bought by more subprime borrowers with more and more unconventional mortgages - until finally there were enough defaults to actually cause problems.
The last straw was probably because a gas price hike made the commute expensive enough that people commuting between big cities and the "executive homes" tightly clustered in former farmers' fields a two-hour commute away from their job could no longer afford both the gas and the payments.
So enough mortgages defaulted that some of the bonds were doing worse than expected. So the demand for them went down. Oops! The positive feedback loop was still in place and it finally got a signal strong enough to get it out of saturation. Lower demand -> lower price -> higher risk estimate -> lower rating -> lower price. Rinse and repeat. Prices for mortgages drop, interest rates rise, more defaults, more positive feedback.
And thus the subprime mortgage market collapsed.
(Then the government throws a trillion or so of our money into pumping it back up...)
Now stock market guys are used to this sort of thing: It's the old chartist vs. value investor dichotomy. Every so often somebody finds a function that predicts the market pretty well from its past behavior, starts trading on it, and makes some money. Then he lets others in on the system, creates a positive feedback loop, a bubble (or crash) forms, the market eventually saturates, it pops (or bounces back), and the now-failed system is abandoned. The later investors lose money to the investors with enough money, wisdom, and a long enough time horizon to trade on their good estimates of the actual value of the underlying stocks. Similarly with commodities (with other pathologies thrown in).
But the bond market investors were used to instruments based on single underlying things - like a big company that borrowed some money - that could be reliably rated. When the rating companies suddenly got themselves into a chartist mode (without realizing it) on these new things they didn't recognize it either.
The cute thing about this is that, once everybody stops using the formula to estimate risk, it will START WORKING AGAIN. And then stop working once more than a few start using it. So be ready for a possible series of similar bubbles now and then as somebody finds another way to apply this, or similar hacks, until enough investors understand what's happening to discredit them as soon as they pop up.