...but I still think the people who keep the market even remotely stable and the people who make the market useful for it's true purpose (giving corporations a bond market and investors a place for potentially stable returns) are long term investors who follow the data.
The problem is that without the people who are speculating and taking bets, you wouldn't have nearly as many people that could make serious investments in companies. People invest in companies (in the sense that you want them to) because they expect them to grow. This is all well and good until an investor decides that he's done investing in the company. Maybe he's less certain about the future growth of the company. Maybe he just needs to free up capital for something else. Whatever the reason, he needs to know before he invests that, when he's ready to pull out, there is someone who will buy up his shares. This is where speculators come in handy. Because they are concerned with the short term effects on stock price and not long term effects on growth, there is generally someone willing to buy what the investor wants to sell, regardless of why he wants to sell. And because there are people constantly buying and selling, because there is a wide market, the investor doesn't have to worry (under normal circumstances) about a shortage of buyers driving down the price: If the market believes there is a true worth to a share, the sale price will be much closer to that price than if there were a small number of sellers who might hold out for a better price because the seller has no real alternative. The more buyers and sellers, the smaller margins people will be forced to accept between the price they buy/sell for and their own perceived value of the stock. Moreover, outside of certain exceptional circumstances, this should lead to a certain amount of price stabilization, as unless something happens to affect the true value of the share (i.e.. new information to suggest that the long term profitability of the company has changed), you have a wide arrange of buyers and sellers ready to buy/sell at a continuum of prices. The price of a share won't drop a dollar if there are people willing to buy the share for $0.25 less than the last sale.
The problem is that a large percentage of the market is speculation, and so the meaning of the stock price has changed. It has morphed from a measure of the value of the company to a measure of the perceived future value of the company to a measure of the perceived future value of the stock price. Of course, the underlying value (and expected future value) of the company still affect the stock price, but when your goal is short term profitability instead of long term allocation of capital, you are forced to be predominantly concerned with share price dynamics, which means that there are horrible feedback mechanisms which cause all sorts of undesired effects.
Of course, this is a problem with how the market works, devoid of any of the newer concerns like algorithmic or high frequency trading. There are plenty of things to be said (on both sides) about their value, but that is outside the point I want to make, which is this:
There are benefits to having some amount of speculation in stock markets, as they add liquidity, which makes non-speculative investors more willing to participate in the system. Problems arise when there is too much speculation, but that doesn't mean all speculation is bad or unhealthy. Unfortunately, putting limits on speculation is hard, and unlikely to ever be done in an optimal way. This is the natural outgrowth of that speculation, and while I personally believe it is more harmful than some other computer-based techniques (e.g. instead of algorithms which identify and then participating in the emotional binges in the market, algorithms which identify when the market has been irrational and set themselves up to profit when they course correct), it is a part of a necessary evil.