Stock-Picking Computers

eldavojohn writes "A while ago, Slashdot ran an article on Algorithms used to augment or replace analysts. Today, the NY Times is running an article on stock-picking computers with quotes from the lovable Ray Kurzweil." From the article: "'Investment firms fall over themselves advertising their latest, most esoteric systems,' said Mr. Lo of M.I.T., who was asked by a $20 billion pension fund to design a neural network. He declined after discovering the investors had no real idea how such networks work. 'There are some pretty substantial misconceptions about what these things can and cannot do,' he said. 'As with any black box, if you don't know why it works, you won't realize when it's stopped working. Even a broken watch is right twice a day.'"

Efficient markets

[Registered Coward v2]

While the idea of stock picking algorithms is neat; market history suggest it won't work a a way to predict performance. What would be interesting is to better search for arbitrage opportunities to exploit faster than others. Of course, eventually others do the same and it becomes an arms race.

Re:Efficient markets

[$RANDOMLUSER]

The only thing an algorithm is guaranteed to do really well is join a panic and dump shares, thereby increasing the panic.

Re:Efficient markets

[31415926535897]

I doubt there are (m)any algorithms that trade like this. This behavior you recognize is actually a function of the way investing is set up. The fundamental problem is that most investments occur while leveraged (so you're trading on borrowed money). If you lose money past a certain point, every broker will demand that you fund your account to the proper level, usually back to the original ratio. Just about everybody trading with leveraged money does not have this extra capital lying around (they ought to be investing it), so they have to liquidate a percentage of their holdings to cover the margin call. This can lead to a negative feedback loop to where everybody is losing money, and liquidating their portfolio probably does not happen by choice for most people.

As another poster has already mentioned...read the story of LTCM [wikipedia.org]. The notable quotes from the article are

In the end, LTCM's basic idea was correct ... but only after the firm was wiped out. Nonetheless, the incident confirms an insight often (though perhaps apocryphally) attributed to the economist John Maynard Keynes, who is said to have warned investors that although markets do tend toward rational positions in the long run, "the market can stay irrational longer than you can stay solvent."

Re:Efficient markets

[SpinyNorman]

It's quite possible that there's so much program trading going on, that you may be able to predict the market (not prfectly, but at least profitably) by effectively predicting what the program traders en-masse are doing. Not that this is necessarily any easier than predicting the mass psychology behind the markets in general, but it does as least point to the fact that much of the market moves according to very deterministic forces.

Anyway, it's already being done, ergo it's possible!

Re:Efficient markets

[mcrbids]

It's quite possible that there's so much program trading going on, that you may be able to predict the market (not prfectly, but at least profitably) by effectively predicting what the program traders en-masse are doing. Not that this is necessarily any easier than predicting the mass psychology behind the markets in general, but it does as least point to the fact that much of the market moves according to very deterministic forces.

And this fact is EXACTLY what stabilizes the market!

As soon as there's a discernable pattern, somebody's going to exploit that pattern in order to make more money, and as soon as that happens, the original pattern gets interrupted, thus stabilizing the marketplace. Perfect? No. But damned good. Some regulation is needed to keep these market forces from being overwhelmed - but the cost of this regulation is a pittance compared to the benefits gained!

Money is an awfully effective invention for distributing wealth, which is why the Star-Trek "utopia" where nobody needs money is not going to happen anytime soon. So long as there is differentiation between different people (and thus resource distribution potential) there will be money.

Re:Efficient markets

[Shisha]

s soon as there's a discernable pattern, somebody's going to exploit that pattern in order to make more money, and as soon as that happens, the original pattern gets interrupted, thus stabilizing the marketplace. Perfect? No. But damned good.

People seem to forget that stock prices do, at least occasionally, reflect reality. So the "original pattern" might not get interrupted. Imagine the classical example of company that pretend to have found gold. At some point the truth comes out and the stock price of the company has nowhere to go but down.

Which reflects the wider problem with computers trading. They do not take reality into account. Maybe once you have an AI engine that can crawl the internet and other news sources faster then humans, then you might be onto something.

Re:Efficient markets

[EastCoastSurfer]

Maybe once you have an AI engine that can crawl the internet and other news sources faster then humans, then you might be onto something.

If you RTFA, this is exactly what they are doing/trying to do now.

Re:Efficient markets

[TheLink]

"Money is an awfully effective invention for distributing wealth"

Yes, but it is even more effective at concentrating wealth.

There are just so many cows and sheep you can accumulate and keep ;).

Re:Efficient markets

[khallow]

Yes, but it is even more effective at concentrating wealth.

While you probably didn't intend it, that is one of the benefits of money and banking. Back two thousand years ago, almost no one could plan ahead 20 years except the wealthy. Any savings you might have accumulated could easily be stolen. IMHO, that was one of the reasons land was so valuable. Someone couldn't break in and take it and it still had use even if everything on it was destroyed. Now, anyone can concentrate wealth, not just the extremely powerful.

Re:Efficient markets

[khallow]

While this thread has passed all bounds of decency or on-topicness, I do happen to think that money still is far more effective at distributing wealth than concentration. First, money forms a small part of the actual wealth out there which also consists of real estate, labor and education, equities and other securities, savings, etc. Sure one can often trade real estate or stock directly for someething else, but these things aren't in themselves money (unless you're about to claim everything with value is money). Second, money reduces transaction costs. That's it's true value.

I claim money actually reduces concentration of wealth. By reducing transaction costs, it reduces barriers to entry for new businesses and the day to day costs for people with little or no wealth. In the developed world, that is a key factor in reducing wealth disparity. There are many factors by which the wealthy can get concentrate more wealth (eg, corruption of government, rent-seeking), but the presence of money isn't one of them.

Re:Efficient markets

[meringuoid]

Money is an awfully effective invention for distributing wealth, which is why the Star-Trek "utopia" where nobody needs money is not going to happen anytime soon.

The communist ideal of Star Trek, as I see it, is possible only because of the replicator.

Consider what happens when it is possible, given an example of a given product, to duplicate it en mass at near-zero cost. Suddenly nearly everything's free. Manufacturing costs nothing, so it's all in the pattern that tells the replicator what to build. This makes it difficult for the capitalists to retain control. They can attempt to retain monopolies on their own replicator patterns, but once the workers, owning their own replicators, begin designing their own replicator patterns, and sharing them amongst themselves and building on each other's work, then the capitalists could end up totally sidelined, made irrelevant by a proliferation of free replicator patterns, put to use to create free products. Would the workers' collective creations be as good as the capitalists' refined and expensive designs? Probably not in every case, but they'd be good enough, and it's hard as all hell to compete with free.

This is happening in one segment of the market right now, and the capitalists there are getting desperate: look at Trusted Computing and the idea of Software Patents, as the bosses try desperately to wrest back control of the means of production. Sadly it can't go all the way; the workers distributing their free software still need to eat! But if we could duplicate material goods as easily as we duplicate software, we could GPL the entire economy and let the workers' socialist revolution really get going.

Re:Efficient markets

[ErroneousBee]

The technical name for this is Technical Analysis, and its a load of bunk.

Sure you can create programs that handle arbitrage opportunities, or detect shortterm effects (market movements lasting less than 1 hour), and these make lots of money for those lucky people who have realtime prices and no brokerage costs (I.e. investment banks, etc).

Stock prices for a company will move on news. Prices may drift around on speculation, but eventually a company will post its trading figures and you will know exactly how much that company is worth at that point in time. Unless these technical analysis programs know which comanies are moving product, who is about to sue who, which companies are in secret negotiations, what the future price of oil will be, etc, then they are going to miss price movements caused by events external to the markets.

Re:Efficient markets

[RKBA]

Exactly what I was thinking. In order to make any successful long term stock market predictions (days or months rather than minutes or hours), a model would have to incorporate worldwide Socio-political and economic modeling. Not likely to happen anytime soon methinks.

$1 says it won't work.

[camperdave]

Trader 1: It's hit rock bottom. Come on, let's buy.
Trader 1 (on the phone): Buy May belly contracts at...

STOCKBOT:- That's a big mistake, Sir.

Trader 2: Why shouldn't we buy now, STOCKBOT?

STOCKBOT:- The price is going to keep going down.

Trader 1: Randolph, this isn't Monopoly money we're playing with.

Trader 2 (on the phone): This is Randolph. Hold that belly order a moment.

Trader 2: Tell me why you think the price of pork bellies is going down.

STOCKBOT:- It's Christmas time. Everybody's uptight.

Trader 1: Could we please buy now?

STOCKBOT:- If you want to lose money go ahead.

Trader 2: What are you trying to say? Please Explain.

STOCKBOT:- Pork belly prices have been dropping all morning. So everybody's waiting for them to hit rock bottom so they can buy cheap. The people with pork belly contracts are thinking, "Hey, we're losing all our money and Christmas is coming. I won't be able to buy my son the GI Joe with the Kung Fu grip, and my wife won't make love to me 'cause I ain't got no money." They're panicking, screaming, "Sell, sell." They don't want to lose all their money. They are panicking right now. I can feel it. Look at this graph.

Trader 2: He's right, Mortimer, my God, look at it.

STOCKBOT:- I'd wait till you get to 0.35 then buy. You'll have cleared out all the suckers by then.

Trader 2: Do you realise how much money he just saved us?

Trader 1: Money isn't everything, Randolph.

Re:Efficient markets

[legoburner]

A study has already been performed to examine basic computer-stock picking based on all available (standard) market data and patterns. I forget who did the original study and am too lazy to look it up but it was covered in 'investment management' by Prof. Stephen Lofthouse. Basically, the programs do not beat the market, but lower the standard deviation (risk) involved in trading. The book recommends the programs are used and refined but states that they are nowhere near being able to be anything more than another advisory tool in the belt of accomplished traders.

Re:Efficient markets

[Courageous]

Don't beat the market!! **Chortle, chortle** A have a good friend who works for a programmed trading firm. They make more than 15% year-on-year, some years substantially more, regardless of the direction of the market, consistently--not on average, consistently!--they've never had a down year, EVER. He's been there 15 years now.

While he cannot say the nature of the programmed trading algorithms, he does not that all their best years are when the market goes DOWN, because their algorithms are better picking good times to short than vice versa.

C//

Re:Efficient markets

[Gorobei]

Basically, the programs do not beat the market, but lower the standard deviation (risk) involved in trading

Um, that IS beating the market: all investments are a combination of risk and expected reward (e.g. treasuries are low-risk and low-return, junk bonds are higher risk and higher return.) If you can reduce risk without reducing return, you can make buckets of money (people will line up outside your door wanting to give you capital.)

That said, don't trust any academic studies on this topic. There are probably only 50 people on the planet who actually deeply understand this topic, and they aren't talking. E.g. when I was running a hedge fund in the early 90s, most of the techniques described in the article were already widespread (yes, we had real-time newswire scrubbers running back then.)

As the article touches on, speed is important: a 5 millisecond advantage lets you make real money without needing to predict anything. The predictors do make excess profits, but most either don't understand the risks they are taking, or actively hide those risk from their investors. D.E.Shaw, mentioned in the article, basically blew itself up back in 1999 or so. LTCM did the same thing.

Re:Efficient markets

[bcrowell]

While the idea of stock picking algorithms is neat; ...
The article is actually talking about at least two different different techniques:

1. Using neural networks to spot unnoticed correlations between stocks.
2. Finding more efficient ways to react to market news, e.g., by using AI to convert a news article into machine-readable form faster than a human can read it.

Only the first one is really a stock picking algorithm.

...market history suggest it won't work a a way to predict performance.
Or you could say that the job of technical analysts is to make an inefficient market work more like the idealized efficient market. Sixty years ago, people used to use various graphical techniques to try to do technical analysis; it probably worked, until enough people did it that it started to eliminate the very correlations that they were trying to detect. Then computers became able to detect more subtle correlations, but by exploiting those more subtle correlations, they removed those as well. Now the arms race continues, and the whole process is presumably a good thing, because it's getting rid of more and more of the effects of inefficient and irrational behavior, and makes the market more and more like the textbook idealization of the efficient market.

Re:Efficient markets

[vertinox]

While the idea of stock picking algorithms is neat; market history suggest it won't work a a way to predict performance.

I sort of thought up how one would basically create such a program. You couldn't simply write one but you would have to "evolve" it.

First take a computer program and have it randomly pick One million criteria from Google news. Then based off that criteria, randomly pick and choose a stock to buy or sell. Now repeat a few billion times. Then kill off the programs that fail to "virtually" make any money (you would not be able to make money off this for a while).

Then duplicate the programs that "made" money and then have each of the billion program have code to modify the criteria and or subtract or add how many criteria you are going to look at on Google. (I mean we don't know if the optimal information the program has to look at is less or more)

So rinse, repeat, and eventually after a few hundred trillion instances you will eventually evolve a program that can look at the information on Google and decide what stocks to pick.

However... My idea will take a very expensive super computer and a direct connection to Google data center so I think the only people who would pull this off would be Google themselves...

Hey wait a minute!

Re:Efficient markets

[scum-e-bag]

isn't that a genetic algorithm?

Re:Efficient markets

[istartedi]

If the Efficient Market Hypothesis were true, stock pickers like Cramer should have been driven out of the market by now. Some investors do, on average, beat the market. See Warren Buffet. Now. The hard part is figuring out if your analyst is the next WB, or just some MBA who isn't too stupid and had good luck on top of not being too stupid... for the last 5 years until he regresses to the mean for the next 10 years. So. If you could write software that picked *analysts* then maybe you'd have something. Trouble is, they're humans. They get divorced, get sick, die, decide they've made enough and want to drop out and do Yoga, etc. Anyway. Bottom line? Not everybody agrees that the market is efficient. I know I don't. No way. Not by a long shot. It's like poker. You'd think that skill is only a small part of the game, and that it's mostly chance. For many players, it is... or at least that's what I thought until I saw a truly skilled player take out an entire table. It was amazing to watch this guy know exactly how to play every hand. Amazing. Never saw it before. Might not see it again, since I'm not a big poker guy. I suspect investing is a lot like that--there are a handful of guys that belong in the investing equivalent of the big poker tourneys. Everybody else is a loser.

Re:Efficient markets

[cayenne8]

"Yeah, if that worked you might as well try and train it to predict lottery numbers. It would be just as successful."

You know...I've wondered why that would not exactly be a good thing to do? I mean, the balls used are physical objects, subject to imperfections. Perhaps these imperfections would select towards some balls being picked more often than others. I was thinking of this with reference to some past 'schemes' where people in casinos kept history of what numbers came up on roulette wheels....after crunching a LOT of data....they saw trends where some numbers were select more often...due to imperfections in the wheel. I would think if this affected the randomness of the roulette wheel....why would it not affect the randomness of the lottery balls?

Well it makes a change

[Silver Sloth]

to see stockbrokers being made redundant by machinery.

Quite futile

[Anonymous Coward]

Discssing this on /. is futile...

Anyone who really knows anything about this subject wil not post. Too much going on in trading land...

Hence AC post...

Re:Quite futile

[Subcranium]

>>Anyone who really knows anything about this subject will not post. ...The only sensible post I've read. I do this for a living, and have been for a while. Discussion is futile because systems traders won't say anything useful. We will say nothing, or we will try to mislead you. We are interested in the $$$ and not the bragging rights. You don't believe me and I don't give a damn that you don't. So this is the stupidest thread ever. I can't believe the dumb things I'm reading. My eyes are rolling so hard I'm getting a lens burn. I encourage you people to make neural nets and join the fray. I'd love the extra money.

Re:Quite futile

[GoofyBoy]

>We will say nothing, or we will try to mislead you.

So in this light, how am I suppose to interpret the rest of your post?

Re:Quite futile

[UbuntuDupe]

I disagree. The most sensible advice can be freely given, because so few people will listen to it. Here is the advice:

1) Don't try to beat the market through picking individual stocks.

2) Buy only index funds. (An index fund is a completely transparent mutual fund that just buys what's in an index, such as the S&P 500, rather than relying on some "hotshot" fund manager who uses a proprietary screening process or something like that.) After fund fees, stock index funds* beat something like 80% of actively managed mutual funds, and more if you factor in taxes.

I always though that, since trasparency has an advantage here, investing would be a good project for the open source community to tackle, but places like vanguard.com already do a tremendous job of it. They offer access to a huge variety of index funds, each of which will tell you exactly what it's investing in (so you don't have to trust any fund manager's expertise), with low fees and low minimums (only $3000 to start one).

*Actively managed bond funds are tolerable if it's in a tax-deferred account with limited choices like a 401k, since taxes are on exchanges are zero there, and the difference is smaller.

Seems only reasonable...

[Thansal]

If you have a giant set of data, and you set a computer on it for long enough, it should be able to come up with some rather solid paterns/corelations/etc. You come up with these things, and if you don't have to wory about paying the stock broker what ever cut for each transaction (because you are the broker), then constantly trading on these thigns seems like agreat idea.

However, it still can't predict things that a human can (yet). I doubt that a computer can incorporate thigns like global news, company announcements, and other such real world variables into how it makes judgments. That was the one thing that the article didn't really talk about.

So I doubt we will see these "black boxes" replacing brokers, simply suplimenting them.

Re:Seems only reasonable...

[timeOday]

I doubt that a computer can incorporate thigns like global news, company announcements, and other such real world variables into how it makes judgments.

Here's [psu.edu] a guy who incorporated yahoo message boards in his stock market prediction software a few years ago.

Actually I was surprised how few references I could find to this sort of thing. Still I don't believe this is an indication that it's not happening; rather, I think market prediction is a black art because investors don't want anybody else to know what they're doing or how they're doing it.

Re:Seems only reasonable...

[An Onerous Coward]

If I had to guess, I would say that NLP techniques are probably mature enough to take stories off a newswire and determine whether the story is good news or bad news for company X (with perhaps 80% accuracy). So, if your goal was to take advantage of jumps and dips caused by news before your competitors, you might have the computer make a few trades.

[Note: the following example is based on my understanding of the stock market, which is most likely wrong]. For example, say you had 1000 shares of Google, currently selling at $432.10/share. Your computer gets a story off the news feed saying Google has announced it will exceed its previously projected earnings, and the computer correctly interprets it as Good News. The computer might automatically place two trades, one which tries to buy twenty shares at $432.10, and tries to sell twenty shares at $433.10. It might do this even as it passes on the news to human analysts, who have to determine whether the news requires more drastic action. They might decide that their system has already adequately covered the new position, or that it will probably drive the price up by $25/share instead of $1. If the computer's rather conservative trade is adequate, then presuming they were among the first traders to act on the news, both trades will probably be successful and they've made a quick $20 without actually changing their overall portfolio.

So I think that while it would be a bad idea to have the computer make long-term strategic decisions about the portfolio, it might be able to make small reactions quickly to take advantage of short-term reactions to news.

Re:Seems only reasonable...

[greg1104]

[Note: the following example is based on my understanding of the stock market, which is most likely wrong]

Completely, but it's good that you know what you don't know--you'd lose a lot less money trading that way. It's impossible to parse stock news and figure out if it's good or bad. Using your example, if Google announces that it will exceed its previously projected earnings, it's just as likely the stock will crash as skyrocket. This is because market participants are working with earning estimation at a higher level than any current generation AI could possibly understand. Read up a bit on whisper numbers [investopedia.com] to get a feel for the challenges here.

Regardless, the notion that you as an individual could possibly get news and act on it faster than the people that really move the market around is naive anyway. Many of these announcements are made outside of trading hours, and after-hours trading is difficult for an individual to do. By the time the regular market opens up again, the big move is already done.

Re:Seems only reasonable...

[An Onerous Coward]

I certainly wasn't suggesting that this would be a promising route for an individual investor. In my mind, it would have required the ability to make transactions nearly instantaneously, and also relies heavily on statistical tendencies (which a big company would be in a better position to handle than a small company).

I still have a hunch that it might be possible to do something in this field. Take product ship dates. While there may be occasions where announcing that you're going to let a deadline slip might be good for your stock (if the conventional wisdom says that you're rushing a crap product out the door), it's more likely to hurt your stock. Even with whisper numbers, it seems to me that (on average, over a broad range of stocks) either the whisper numbers are going to stay close to industry generated forecasts, or there is something seriously wrong with one of the two sets of numbers. So a revision upwards should be more likely to bump a stock than to tank it (though it might be a close thing).

Another thing to remember is that the goal is to act quickly on the information, before the market has time to fully digest the implications of the move. It seems like first impressions would matter a lot in such cases. You know, simple metrics like "increased earnings == good", "layoffs == outstanding", "merger == cha-ching", "merger with layoffs == do-money-dance". Hopefully you can get the trade done before the market does a doubletake and says, "You know, maybe Google buying YouTube isn't the smartest thing in the world." A basic system seems like it would have a better chance at predicting knee-jerk movements rather than the ones resulting from a few more moments consideration.

Last thing: If you're a company that is already buying projections, it doesn't seem too hard to take the results of the initial news-reading system and turn over the data about the new forecasted earnings to another system that already has knowledge of the projections. While the first system makes a small trade, the second one might decide to make a larger one if the new numbers beat the forecasts, or try to cancel the trade made by the first system.

This is why I don't play the market myself. I'd have to feel pretty self-confident to think that I could consistently outperform the market as a whole. Index funds, mutual funds, whatever minimizes the actual thinking I have to put into managing investments. My school brought in a guest lecturer who was a major honcho at an investment bank that does a lot of computerized training. I got some interesting stuff from him.

1) people try to sell his company on new algorithms almost daily. Many of them only appear to work because the author hasn't accounted for something (sometimes even things as small as network latency).

2) most of his research was focused on using current trading data to make short term projections. So throwing in external information sources might still be an exploitable niche.

3) Wall Street types are obsessed with money, and even being in the room with them for an hour makes me feel dirty.

Between #1 and #3, I'm positive I would never try to build such a system myself. Thanks for the information. It's a definite wake-up.

Re:Seems only reasonable...

[greg1104]

So a revision upwards should be more likely to bump a stock than to tank it (though it might be a close thing).

Market theorists love to point out with much amusement that a lot of things that seem like they would be strongly associated with a clear move in one direction turn out, after you analyze them properly, to be close to 50/50 that you can't trade on them usefully.

It seems like first impressions would matter a lot in such cases. You know, simple metrics like "increased earnings == good", "layoffs == outstanding", "merger == cha-ching", "merger with layoffs == do-money-dance"...A basic system seems like it would have a better chance at predicting knee-jerk movements rather than the ones resulting from a few more moments consideration.

Let me pull apart one of those as an example. If you watch a few of them, merger announcements usually have a strong impact on both companies involved. Even people have a hard time distinguishing whether a buy-out of a company is a bargain, or a big overpayment relative to the true value of the company. With the same big news announcement, one company's stock will go up, and the other's will go down, all based on the perception of whether the terms of the merger gave one party a better deal than the other. So we have one annoucement, which is likely to increase the price of one stock and decrease the other; it's a very complicated thing to predict.

And how the market will react to layoffs is dramatically more complicated even than that--layoffs as a cost-cutting measure might be good for the price, layoffs because the business is dying might kick off a gigantic selling spree, and the price can jerk around all over as big-name analysts weigh in with their opinion over the days that follow the initial news release.

I went through this whole excercise myself for a while, trying to get a computer to parse the news and do something useful with it. Turns out, there's a much easier way to figure out whether the market views something as good or bad news: just watch what the price of the stock does. You'd be far better off learning some of the techniques of technical analysis than bothering trying to understand whether the tone of a news pieces is good or bad. It would take years of training a sematic-based approach before it had any hope of being better than an approach that just looks at a few minutes of where the market is moving and assuming it's going to move in that direction more; a simple moving average crossover technique works just fine for that.

Re:Seems only reasonable...

[An Onerous Coward]

Don't be sorry. I think I see the problem [wikipedia.org]. In a nifty sort of self-referentiality, word sense disambiguation is also an NLP problem.

In this case, NLP means Natural Language Processing. This is the process of taking information written in a human language (English, for example) and using a program to extract useful information out of it. It's a step that would absolutely be required for the rest of my system to work, since all these announcements the computer would be required to make decisions on are written in human language.

In this case, the program would need to read press releases and extract the following useful information: The stock to buy/sell to take advantage of the information (which goes a bit beyond simple Named Entity Recognition, because other companies might be named in the bulletin, but should still be relatively easy) and whether the stock is more likely to go up or down based on the news. The latter task is obviously more tricky, but it would definitely be an NLP problem.

Spam!

[NineNine]

and other such real world variables into how it makes judgments.

It also can't take into account stock-pumping spam, which has graced so many of our inboxes lately.

Humans *really* bad at predictions.

[Colin Smith]

However, it still can't predict things that a human can (yet).

People are really bad at predictions. Managed funds rarely even keep up with simple index linked funds.

I doubt that a computer can incorporate thigns like global news, company announcements, and other such real world variables into how it makes judgments.

No this really should be pretty simple. We already have an algorithm that can classify a message as spam or not spam with 99.5% + accuracy, basically good or bad. The same algorithm could be used to classify any news article as good or bad for a particular stock by looking at the direction the stock takes after the news appears. You could even train the classifier automatically by looking a couple of days in the past. Simply classify every single financial announcement against every single stock, eventually it'll get to the point that the classifier will be able to say that an announcement is 80% probably good news for this stock, 92% bad news for that stock.

   

Re:Humans *really* bad at predictions.

[greg1104]

The same algorithm could be used to classify any news article as good or bad for a particular stock by looking at the direction the stock takes after the news appears. You could even train the classifier automatically by looking a couple of days in the past. Simply classify every single financial announcement against every single stock, eventually it'll get to the point that the classifier will be able to say that an announcement is 80% probably good news for this stock, 92% bad news for that stock.

If you ever tried to build such a system you'd realize the problem is exactly as hard as predicting the motion of the market as a whole. Market participants are working with earning estimation at a higher level than any current generation AI could possibly understand. Read up a bit on whisper numbers [investopedia.com] to get a feel for the challenges here. You could have an earnings report that says "earnings are up, we just increased sales, and we rule!" that doubles the price of one stock, because no one was expecting that. Meanwhile, the exact same report from another company tanks the price because people had driven the price of the stock up before the news in anticipation of even better numbers.

One trade I went through this with stands out in my memory. I had what I thought was juicy new information about recent sales trends for a company, and I purchased many shares before their earnings report expecting that sales were going to be up 40% from the previous quarter. Sales were actually up 50%. The next morning, as the market opened, instead of counting my profits I discovered the stock had dropped 15% after the announcements. Turns out, people with much better information than me had driven the price up over the previous few weeks in the expectation that sales were actually going to be up 70%. The notion that you can compete with this sort of behavior after the news announcements has come out is at best a wonderful fantasy; at worst, you'll actually try to do it and get crushed.

Re:Humans *really* bad at predictions.

[Colin Smith]

urns out, people with much better information than me had driven the price up over the previous few weeks in the expectation that sales were actually going to be up 70%. The notion that you can compete with this sort of behavior after the news announcements has come out is at best a wonderful fantasy; at worst, you'll actually try to do it and get crushed.

A single announcement wouldn't be used that way on it's own, it's simply a way to condense a text announcement down to probably good or probably bad, it means you can use it as one of many possible indicators for a few tens of thousands of genetic algorithms. You'd be classifying on every announcement against every stock to provide a decent range of information. The genetic algorithms which give too much or too little weight to the announcements will predict badly and fail to thrive.

 

Re:Humans *really* bad at predictions.

[greg1104]

A single announcement wouldn't be used that way on it's own, it's simply a way to condense a text announcement down to [an indicator]...The genetic algorithms which give too much or too little weight to the announcements will predict badly and fail to thrive.

I guarantee you that if you put some serious time into this approach, you'd discover that the weight of any "announcement tone" indicator would end up so close to 0 that computing it in the first place would be a waste of time. Here's a better indicator: watch what the price of the stock does for the first few seconds the market is open after the announcement. Whatever direction it moved in, there's your answer for whether the announcement was good or bad from the perspective of stock price. If you put those two indicators in a genetic optimizer cage max, the fancy sematic news-reading approach is going to be left in a puddle of blood every time, squashed by a stupid but very strong indicator whose sole mantra is "the market moves the way it moves".

Technical analysis

[nuggz]

But if someone else uses the same algorithm and beats you to market on your trades by even a fraction of a second your information is outdated.

If it was really possible to predict price behaviour on trading patterns these opportunities would be fully exploited until such a prediction would not work.

Why not simply buy a good profitable company?
Or a company that will be good and profitable?

Re:Technical analysis

[nuggz]

If a company is good and profitable and the price is less than the intrinsic value.
Eventually the market will realize this. If the market undervalues a stock it leaves an opportunity for you to scoop it up nice and cheap.
Similarly if the price is above what the company is worth, go ahead and sell, but something more attractively priced.

One alternative is to buy companies that give their earnings to shareholders, then it doesn't matter if the market realizes the value, you'll get your profits as they happen. This is one reason conservative dividend investors tend to do so well.

Re:Seems only reasonable...

[Mad Marlin]

The problem with trading on news reports is that by the time you read about it in the Wall Street Journal, that is, by the time a newspaper reporter is aware of it, the news is already factored into the stock price.

Re:Seems only reasonable...

[TwistedSquare]

If you have a giant set of data, and you set a computer on it for long enough, it should be able to come up with some rather solid paterns/corelations/etc.

That is an invalid assumption. The computer will only pick up solid patterns in the data if they are there to be found. If there is a solid (useful!) pattern in the market, a computer may pick it up. But the question of whether there are such patterns in financial markets is just as important (if not more so) as whether we can find them with a computer.

Re:Seems only reasonable...

[greg1104]

If you have a giant set of data, and you set a computer on it for long enough, it should be able to come up with some rather solid paterns/corelations/etc.

If by "solid" you mean "tradable" then no, it won't. It will produce what trading system developers refer to as a curve-fit trading system. When exposed to new market data, that type of system really doesn't work well at all.

Think about this for a minute...how well would a trading system trained on data about the .com boom work in today's market? How well would one trained on the .com bust work? Pretty bad, right? Well, guess what...you never know which type of market you're in, and the market is constantly changing. Now you're thinking "then just train against enough data that you've seen everything"...ha! Do you think that today's market, with tons of electronic trading at very low prices, move exactly the same as the ones from the 70's where individual trades costs hundreds of dollars to execute? You can either have a small amount of data that reflects how the market is working recently, or a large amount of data that's mostly about a market that doesn't work the same way the current one does; it's impossible to get a lot of data on how the current market moves, because the current market only exists a tick of data at a time.

People who actually build useful systems start with a theory about how prices move, then write code to analyze the market to see if that strategy is tradable. I would suggest you read an article by well-known traders William Eckhardt and Richard Dennis at http://www.visoracle.com/swingtrend/optimization.h tml [visoracle.com] as an initial bit of food for thought; a search on curve fitting in trading systems will lead to many, many more article on the subject of why having the computer just look for patterns on its own is of extremely limited value.

This would be my request...

[bogaboga]

I wonder whether there are Open Source or Free-to-use programs to trade currencies. I tried once and got "burned" hard...almost lost everything!

Most companies provide free trading stations but the tools/software to predict what might happen in the market cost thousands of dollars, yet these tools are never 100% accurate. So, does anyone know where one can grab Open Source versions of these? Thanx.

Re:This would be my request...

[31415926535897]

The system you are looking for is Genius Trader [geniustrader.org].

However, while it's fun to play around with a system like this, I must warn you that the realities of trading make it very hard to profit even if it looks good on paper. You probably know this, since you "got burned" before. Make sure you consult a professional before investing, or I can pretty much guarantee that you'll get burned again.

The predators and the prey

[Colin Smith]

Really, if you don't know what you're doing then you are the prey. I don't, and I recognise I'd get eaten by the big guys. Better idea to take up a known to work investment strategy and invest for the longer term. You can still beat the market average and the bank savings rates but you won't be making 100% per day.

There are several very simple to manage portfolio strategies which will give you very good returns whether the markets are going up or going down and with little risk. It's not magic, it's common sense, long term investing and frankly a bit dull.

 

Re:The predators and the prey

[bogaboga]

There are several very simple to manage portfolio strategies which will give you very good returns whether the markets are going up or going down and with little risk. It's not magic, it's common sense, long term investing and frankly a bit dull.

Care to post a link? Thanks.

Re:The predators and the prey

[Colin Smith]

Care to post a link? Thanks.

sure.
http://www.npr.org/templates/story/story.php?story Id=6203264 [npr.org]

My own portfolio is a bit simpler. UK index linked mutual fund, developing country mutual fund, government bonds, commodities (gold silver), housing stocks. Basically about 20% in each sector. Try to spread your portfolio over several sectors which don't all go in the same direction at the same time.

The strategy is simple but it's the important bit because it stops you buying at the top of the market. It's called rebalancing.

Every month add the £100 (or however much you want to invest) to one of the categories. You choose the category by taking an average of how much you've invested so far, divided over the categories. Say after a year you've invested £1200,the average which in my case should be in every category is about £240. But the market goes up and down so some sectors will be doing badly, they'll be below the average, some will be above the average. Well, the whole idea is to buy low, sell high, yeah? You invest your next £100 into the sector in your portfolio which is the furthest below the average, so topping it up. This way you're always buying into a sector when it's low, not high.

Sit down once a month with a spreadsheet (or calculator, it really isn't that hard) and work out where to put the money this month. You spread the risk over 5 or however many categories/stocks you want to take, when the markets change direction, as they inevitably do, your cheaply bought investments will go up most and you are not putting your money into expensive top of a peak stocks.

It's simple and it works, as mentioned in the link the pros do it on a minute by minute basis rather than on a month by month basis (and they sell as well as buy) but I'm really not that interested in finance, and I don't need to be interested or knowledgeable. I'm beating 10% per year for an hour a month.

Oh and always stick the monthly £100 into a single category, don't try to spread it because fees etc will be minimised.

 

Re:The predators and the prey

[abigor]

Buy the book called "The Best Little Guide to Stock Market Investing" from Amazon, open a trading account at an online discount brokerage, and you're off.

Re:The predators and the prey

[geekoid]

Actually, if you cuold tie into the same databases for exchange rates, and move your money with the same speeds as the large players do, you could probably do all right.

Of course, if you could tie your money on a small scale to the movements of the top players you could do damn good.

Sadly, this technology isn't consumer available.

Re:This would be my request...

[antifoidulus]

Shhhh! This is slashdot! Jeez, don't you know that anything with the words "open source" in it is automatically infallible here?

It Works

[SRA8]

Forget quotes about neural networks, program trading WORKS. Firms like Goldman Sachs have pulled in hundreds of millions in profits with program trading and will likely continue to do so. Its about replicating human trader behaviours, except with lightning-sharp reflexes.

Re:It Works

[micromuncher]

People who say its too volatile and not logical are wrong...

If you have level 2 access, you can pretty accurately predict where a stock is going. Sure, you don't know from day to day, but there are reasons for market introduced delays and market makers.

Re:It Works

[balsy2001]

That is not exactly true. I get live quotes from all american exchanges, but if you want Level II quotes from NASDAQ, for example, you have to pay extra for that and you do get more than the quotes. You will get the information for all of the currently placed orders with their limits. Thus you can better predict where a stock is headed. For example if want to buy stock XYZ and you just have a live quote you will only notice that the last trade was made at say $5.55 and you might say I will buy at that price. The level II quote might also tell you that there are say 12 orders to sell XYZ ranging in price from $5.55 up to $5.60 while there are 120 buy orders that range from $5.20 to $5.54. This will tell you that the rest of the market doesn't think the stock is very attractive at $5.55 and will likely go down.

Re:It Works

[balsy2001]

My only point was that Level II was more than just real time quotes. I totaly agree with you that I could not game the system and don't even try. There are so many more advantages that the "big boys" have that small fries like myself can possible hope to out pace them day trading.

Re:It Works

[balsy2001]

Since I didn't actually answer your question: I have used level II quotes but not to try an day trade. I had SAI and was up 20% and wanted to get into another investment. So I placed a limit order at the LAST price. Usually that is good enough to unload but SAI doesn't have a large volume and my order had not gone through after several minutes. I used Level II quotes to find a good price to try and unload. I didn't want to arbitrarily drop the price by $0.2 and Level II showed me that there were orders that would take at $0.03 below my previous order.

Re:It Works

[micromuncher]

I didn't say l2 was not delayed. l2 gives you depth and bredth of bid/ask. If you track time and sales in addition to bid/ask and size - you can get a better idea of up or down trend for the next short time window. And the delay I'm talking about is execution, not real time quote.

Re:It Works

[OneSmartFellow]

That's not the same as automated stock picking.
Automated trading systems 'generally' are used to take a position in a stock that has already been picked.
So, trader A in Goldman Suchs wants to take a long position (buy) 100K shares of IBM, so he assigns that trade to the algorithmic trading engine, which might offer him various algorithms to help fulfill his position at the best possible price, ranging from %vol, VWAP, 'iceberg' or other type of algorithm.

notice, though, that the trader already had the stock to trade chosen, he didn't let the algorithmic engine choose it for him.

It's a prime directive issue

[bennomatic]

There are so many theoretical problems with automated stock picking systems that I could spend all day working on this post. Instead, I'll ignore all but two.

First, no matter how well you can predict based on patterns, when you are picking individual stocks, there is such a huge influence from the chaos of human nature that, from day to day, no matter how appropriate your predictions are (based on history), they may have nothing at all to do with reality.

Additionally, if you get enough of these stock picking systems in operation, they can actually change the dynamics of the market, keeping them from being accurate for years as they all try to account for the activities associated with each others' predictions.

The problem with stocks is that in order to know how they are going to perform, you have to know not only what the company is going to do and how their customers are going to respond, but also how the investing public is going to take that news. It's an odd mix of fundementals and faith, in my experience.

The problem is markets are choice based

[argoff]

While there are probably measurable patterns. Many aspects of a market are choice based. For example, today central bankers are in a real bind. If they raise rates, it could crash housing and the whole economy with it. If they lower rates, there could be a panic out of the dollar to currencies that offer a higher return on interest rates. Once that move starts, than even investments that don't do anything at all (like gold) will rocket, making the panic even worse. In the end a human is making those choices at the helm, and a only a human can have the intuitive understanding that you better buy gold (but not on margin), even if it looks like a crappy investment. Another thing, if someone knows that the computer reads pattern X Y Z as a sell, then they might try to force the stock price to make that pattern against the market to force a computer sell and get in on it at a good price.

You wanna use your computer?

[Colin Smith]

There's moneybee: http://uk.moneybee.net/ [moneybee.net] or maybe wealthlab http://www.wealth-lab.com/ [wealth-lab.com]

Or you could try genetic algorithms, download the info on the whole market plus historical info, give the algorithms access to the lot, plus downloaded financial news, classify the financial news as good or bad for a stock using a bayesian classifier, add that to the pot and then use evolution to see which algorithms survive best in the market to date.

You may need to build a supercomputer to run enough algorithms to perform the search. Look at it this way though... NOW you have an excuse.
 

Mr. Lo is not smart enough to teach at MIT

[plover]

I hate to be so mean, but anyone who turns down a job from a fund with $20 billion dollars is just not smart. He could have milked that for 20 years! He could be pouring grant money over his breakfast cereal right now, but "oh, no, sorry, but that's not technically feasible." Fool. ANYTHING can be made to sound technically feasible for the right amount of money.

[ Yes, I am joking. I'm quite sure Mr. Lo is brilliant -- just maybe a touch too honest. :-) ]

Re:Mr. Lo is not smart enough to teach at MIT

[cperciva]

[ Yes, I am joking. I'm quite sure Mr. Lo is brilliant -- just maybe a touch too honest. :-) ]

You're assuming that he cares about earning lots of money. More likely, he has enough money, and he wants to do work which he finds interesting -- in other words, he's not turning down the offer because he's honest; he's turning down the offer because he doesn't want to waste 20 years of his life.

Re:Mr. Lo is not smart enough to teach at MIT

[plover]

[ Yes, I am joking. I'm quite sure Mr. Lo is brilliant -- just maybe a touch too honest. :-) ]

You're assuming that he cares about earning lots of money.

Wrong! I'm assuming nothing. I'm telling a fracking joke, and everyone here took it waaaay too seriously! I said, "I am joking." You even quoted me above saying, "I am joking."

OK, so I want everyone who posted a reply to my comment thinking I was serious (and that includes the moderators who modded this up as insightful) to stand up and take this pledge:

I JUST READ A JOKE.
THE JOKE EVEN SAID IT WAS A JOKE.
I ACKNOWLEDGE THAT IT WAS A JOKE IMPLYING THAT SOME GUY PASSING UP MONEY IS STUPID.
IN THE FUTURE WHEN I SEE A SLASHDOTTER SAY 'I AM JOKING' I PROMISE NOT TO TAKE IT SERIOUSLY.
IHBT, AMEN.

Sheesh.

It's harder than it looks

[$RANDOMLUSER]

Anybody remember Long-Term Capital Management? [wikipedia.org] They were a hedge fund, heavy with experts, even had a Nobel Economics Laureate on their team. They lost 4.6 billion dollars in four months time, having to be bailed out by the big banks to prevent them from crashing the whole market. PBS (I think Frontline) did a documentary on the whole thing. Fascinating stuff - I wish I could find a link for it.

Re:It's harder than it looks

[$RANDOMLUSER]

Found it: it was Nova: "Trillion Dollar Bet". Here's [pbs.org] the transcript.

Re:It's harder than it looks

[hughk]

One of the key problems of LTCM was slippage, that is the lag between needing to trade and the execution. Slippage can be caused by speed of access to the market (not an issue for LTCM, but a problem for those who do not have direct or near dircet access to the market) and also liquidity. Your algorithm may tell you that you need to buy a secutrity, but there aren't enough to buy so the price jumps too quickly.

Note that LTCM had privileged market access with effective direct backing from major banks. That meant rather than covering their own positions, due to the participation by the banks, it was their money that provided collateral (no longer permitted). In effect, LTCM going would then have caused several major clearing banks to go, and not just in the US but wherever LTCM were trading.

The problem is too big

[gregor-e]

Let's see... we want to design a neural net that predicts the summed trading behavior of, oh, several tens of millions of human brains? Beyond the obvious hardware scaling problems, we must think of the software used by these brains, each of which has different biases for selecting stocks. Some choose based on omens they call 'fundamentals'. Some choose based on some form of scrying on the shapes of squiggles of price history charts. Some trade decisions are made on how the trader's 'gut' happens to feel about it. But, in fact, everything stored in these neural nets has some contribution to the outcome of an investing choice. In a close decision, it may be that the memory of having been turned down on an offer to go to the prom might provide the unconscious final impetus to invest or not.

Kinda reminds me of a fanciful and ambitious kid who declares he's going to build an interstellar spaceship out of stuff laying around in the garage.

Re:The problem is too big

[vertinox]

Let's see... we want to design a neural net that predicts the summed trading behavior of, oh, several tens of millions of human brains?

I hate to say this, but you give the human race too much credit. Those tens of millions of human brains tend to think in herds. Often to the chagrin of the hand full of people who sell short and make a profit.

Playing the stock market is like out running hungry grizzly bears. You don't need to faster than the bear (the market) but you do need to be faster any one else with you (the other investors).

About Markets and mathematics..

[12357bd]

The last work of Mandelbort (the 'fractals' father) 'The (Mis)behaviour of markets' http://www.amazon.com/Misbehavior-Markets-Benoit-M andelbrot/dp/0465043550/ [amazon.com] is quite interesting.
Sigh, markets are chaotic, much more chaotic than current market analisis states.

Odd

[Alioth]

It's odd how online poker is illegal in the United States, but the stock market (which is largely just gambling) is not.

Re:Odd

[hughk]

The stock market isn't just gambling. At least, in theory you are buying a 'piece' of the action. What is a gamble are derivatives. In fact you will find that some derivatives really are being sold by bookmakers, say spread bets.

Linux and Stock-Pick Software

[bruthasj]

This is the last thing that's holding me back from adopting Linux. If it came with stock pick software, I would use it.

Another would be user

[geekoid]

not using linux because it wont run there favorite game.

Re:Linux and Stock-Pick Software

[ChrisMaple]

Linux does not prevent you from using free screeners at Yahoo, Reuters, and Zacks, among others. Yahoo is better for day traders, Reuters and Zacks for longer term.

Buy low; sell high; make money!

[GuinevereTheWhitePha]

In theory, making money in stock market should be easy. Buy low, sell high is a guaranteed successful investment strategy and yet few people follow it. Why is that? By definition, an undervalued stock is an unpopular stock. Doing the unpopular thing is very difficult. Quite simply, human beings are a social animals and going against the crowd is often not in our best interest. Investing in unpopular stocks almost always feels wrong.

However, if you take a rational look at popular and unpopular stocks, you will see that the unpopular stocks are the ones likely to be the real money makers. Consider a mythical corporation that we'll call XYZ Corp. Let's assume that there is a lot of popular interest in the stock. The analysts all believe that great things are in this company's future. Should you invest in it? No, because the stock's price already reflects belief in the company's rosy future. The price of the stock always reflects everything that is known about the company. This concept is important to grasp.

Now, let's consider the future. The future is, of course, impossible to predict, but two outcomes are guaranteed. Either the company will live up to the investors' expecations or it will not. If it does live up to the company's expecation, the price of the stock may stay flat or rise only moderately since the stock price already reflects the high expectations of the investors. If the company fails to meet investor expectations, the price of the stock will drop in response to lowered expectation. Popular stocks don't have very much upside potential in the long run.

Now, let's assume the converse. XYZ Corporation has been mismanaged and had some bad luck. Investors and analysts now consider this stock a "dog." What are the possible outcomes for this stock? Currently the stock price is depressed because it reflects all the pessimism folks have about the company. In the future, either XYZ Corp will continue to run the company poorly or it will change its business strategies and turn itself around. The price of stock will stay flat in the first case and rise in the second case. Unpopular stocks have a great deal of upside potential and little downside since the price already reflects low expectations.

Re:Buy low; sell high; make money!

[geekoid]

which is exactly why large stock money makers pay people to get on a stock BEFORE the general public view it as rosy.

Something that was easier to do 10 years ago then today.

Re:Buy low; sell high; make money!

[ChrisMaple]

The problem with buying low is that some companies do go out of business and the stock becomes worth zero. Others get bought out at a low price (you don't always have the opportunity to say "I won't sell at that price.").

If the price stays flat you lose against "inflation".

You need to do much more analysis than say "This stock's price is low, it's time to buy."

Vitesse (VTSS) got to over $100. You could have later bought it for the bargain price of $10. The most recent close was $1.20.

Program trading is the new day trading

[bigberk]

First of all, the article summary seems a bit misleading because people might think we're talking about "stock picking" as in analyst opinions/monkeys picking stocks. That's not so, this is about software driven trading. Often this is of a very short term nature, so in a way the growing use of program trading is the "new day trading" fad.

In the industry it's called "program trading" and refers to automated, algorithmic trading of instruments such as stocks, futures, forex. This is regularly done by many banks and large funds, and also small investors. In fact there is a discount brokerage which I'll just call IB here, that has an API which lets anyone program their own computerized trading. It's a bit "too easy" to do.

That doesn't mean it's always profitable in the long term, but without a doubt people are profiting at least in the short term. The software has multiple strategies, well documented approaches and algorithms. Generally the trading robot is trying to ride trends.

As someone who follows these things, here are a few criticisms I'm aware of:

1. These short term trading activities require high leverage, because trades have to be for large amounts of money to make them worthwhile. You need large amounts of money to make this work, because things like trading costs eat into profits tremendously. Again, like day trading.

2. High leverage is risky because one big mistake or unpleasant event could wipe out tons of past small gains. Risk management becomes a key issue. Some would argue that perceived risk in markets these days is unreasonably low. Does this unbalance the risk/reward equation?

3. Market-wide, we know program trading has increased dramatically on US exchanges. Add to this the undocumented program trading (smaller traders who don't have to report it to anyone) and basically there are a ton of computer algorithms out there today trading stocks. Everybody can't make money at the same time, so to profit the participants have to use even greater leverage = more risk.

4. Programming flaws, bugs, or improper risk management could have tremendous market-wide implications. Take for example the huge market moves in 1987; the drop was a "20-sigma" event and not anywhere within the realm of possibility back then. Obviously the models failed to handle it. Similarly, the next time we have a "big event" in markets, today's algorithms might fail. If a large number of computers choke while trading, could bad things happen?

5. So under unstable market conditions, the program trading could lead to increased volatility (like daytrading caused volatile markets during the crash). But under stable market conditions, like we have today, program trading seems to smooth out daily movements. Notice that the US markets hardly move as much as 1% in a day; trends are smooth and volatility is extremely low. The VIX, a volatility measure, has hit historic lows.

Working Examples

[micromuncher]

If you think about the number of symbols being tracked from the key exchanges, we have over 250,000. A publish an subscribe system usually bottoms out at 20,000 topics. (In the old days, SmartSockets had trouble with 2,000.)

The number of events/updates comming through, at least 7 years ago, was about 800 every second. That's a lot of information.

So, what can we do with that much information that is updated so frequently?

Well, implementing time and sales for just transactions takes lots of disk space. Gig for a day. Now, try and keep yearly data. Now you have huge, absolutely freaking huge, data storage issues. And search... wow. Neural doesn't seem appropriate.

So where do we actually implement automated trading?
Look for a few commodity stocks, high volume, no dividend. These will follow certain daily trends. Apply some fuzzy logic - threshold and objective functions. In the morning, we are high. We drop until mid day, and then we go up a bit. Drop again in the after noon, and we're usually high or low at end of day more along our daily trend.

And then there is NASDAQ-OTC, or any pink sheeted company. Here we take advantage of stupid people and penny stocks. If the stock is worthless, there is a large float, and there is news, and suddenly there are a bunch of transactions, we can start our automated shorting. News flows a lot less frequently. And you know a dump is comming after a reverse split. This is highly automatable.

Any fund managers want to give me money to write it for them? It has been done before...

Humans can't do it...

[Duncan3]

You didn't think the 370 Trillion in derivatives was traded by humans did you?

Of course, the entire planet's GDP is only 60 trillion, so even a little mistake means complete global meltdown.

.

Re:Why?

[proverbialcow]

First you have to get a license for the exotic pet, then you have to import it, then you have get it vaccinated and take out liability insurance in case it throws feces at a passing dignitary.

No, no, no, there's a better way. Get me a case of beer and a copy of the WSJ and a dartboard. I'm at least as random as a monkey after 24 beers.

Re:Why?

[stunt_penguin]

Personally, I'd be more worried about it throwing the darts at a passing dignitary, or at myself.

Remember: sticks and stones may break your bones but feces just splatters.

Re:Why?

[Xyrus]

After 24 beers, we'd still have to get liability insurance in case you throw feces at passing dignitaries.

On the plus side, we still wouldn't need to get you vaccinated, imported, or liscensed.

~X~

Re:Why?

[meburke]

In some cases, the result seems to work more consistently.

I had a friend who worked in the AI department of Lockheed (about 20 years ago) and they developed the software that was used for the Robert Prector's Elliot Wave newsletter. Every two years they would give the program to a couple of people to try for 6 months. These people would invest $10,000, use the program and follow the guidelines, then evaluate the results. I was privy to the outcomes of three of these tests in the mid-nineties, and the lowest was earned $15,000 and the highest was earned $36,000. These are pretty good results. (However, the stock market was steadily climbing during those years and I wasn't able to compare results with EWT competition. Still, if I was able to consistently get 30%/year on my investments...)

Back in the 70's, Dean Witter had a program called PACE. I know two people who had a system for using it that earned them over $100,000/year, and they never deviated from the program while I knew them.

Then I have a friend who is a very conservative money manager (manages a couple hundred million of other people's money), and over the period that stocks crashed (remember Enron and Worldcomm?) he only had two clients lose any money, and the biggest loss was less than %15. He claims that these programs are mostly bunk. (This guy is a perfectionist, and I bet a computer is no more disciplined than he is.)

These programs are not investment management programs. The principles of investment management are pretty simple. The best book I know on the subject is still Benjamin Graham and David Dodd's book, "Security Analysis". However, the problem is finding opportunities that comply with the principles. Systematic data analysis by computer could have a profound effect, and that's what most of these programs do.

BTW, the article mentions that profits are slowing down: In Robert Prector's book, "The Elliott Wave Theory" and in his newsletter, he sort of predicts that as information becomes more available for analysis trading will be done more rapidly on spreads that may show profits as low as 1.5%

Elliot Waves = Cabalistic Numerology

[hughk]

I am not aware of any real long term benefit in Elliot waves or so called Fibonacci trading. However as with any theory where there are a number of adherants, you can simply follow the flow. This will generally work until there is a major market adjustment.

It could happen

[meburke]

A HUGE amount of the market is in retirement funds or investment funds designed to be liquidated through retirement. My generation will be retiring soon, and withdrawing their funds from the market. If the USA keeps producing, it may be a good place for foreign investment that will buy the stocks being sold. However, there's about 10 Trillion dollars of off-balance-sheet liabilities in Social Security, Medicaid and Medicare out there. (If the USA was Enron or Worldcomm, the last 8 Presidents and all the legislators would be in prison today.) I suspect that the government will fund these liabilities by inflating the currency and raising taxes, making the USA attractive investment opportunity for only China. The gambling schemes, including the derivatives you talk about, will fold. The world will get hungry...invest in food.

I support the investor's right to be as stupid as they wish. A buy and hold regulation won't do as much good as something like the Fair Tax Plan, increased personal debt reduction, a balanced budget, and reduced spending. Presumably, investments are property that an investor should be able to divest anytime. Leave the government out of it.

Re:It could happen

[killjoe]

US has been unable to support the dollar so far. All other currencies have been beating up on the dollar like it was a red headed stepchild.

The dollar is due for a complete collapse. China has trillions of dollars and they are seriously considering insisting on other currencies for payment, same with oil producing companies all of whom hate our guts and would love to see the dollar collapse.

If you have money to invest do it on commodities. Food, minerals, oils, water etc. The smart corporations have already raided the water supplies of most of the world and very soon people in africa, india, and asia will be paying big bucks to companies so they can stay alive.

If you want to diversify invest in the far east.

Re:Why?

[DavidHumus]

This is one of the most beloved urban legends of people who need an excuse for their poor luck.

In fact, in the Wall St. Journal's long-running contest, the experts have out-performed the darts 29 to 21 times (ahref=http://www.webtrading.com/issue18.htmrel=ur l2html-21923 [slashdot.org]http://www.webtrading.com/issue18.htm> ). This does not mean the experts are all that great - the score is 26 to 25 againsT the Dow Jones Industrial Average.

However, in the less-long-running contest of Wall St. Journal readers versus darts, the readers are getting trounced 13 to 9 (ahref=http://online.wsj.com/article/SB11584521400 7665534-search.html?KEYWORDS=dart+picks&COLLECTION =wsjie/6monthrel=url2html-21923 [slashdot.org]http://online.wsj.c om/article/SB115845214007665534-search.html?KEYWOR DS=dart+picks&COLLECTION=wsjie/6month>).

What is true, and more damning to investment professionals, is our poor performance versus broad-based indexes, which are inexpensive investments.

Re:Why?

[0xABADC0DA]

Since the darts should hit with some sort of normal distribution with the "top" of the bell curve being the center of the paper, the lesson learned here is that as the companies symbols tend toward "MMMM" the probability of success goes up. This explains the success of 3M (symbol MMM) and of course my newly formed company "MMMM, Inc". We sell flavored snake oil, and let me tell you investor confidence in the flavored snake oil market has never been higher.

Forget about the monkey

[WebCowboy]

Surely you can get a monkey and a good set of darts for less than what they were going to spend?

Getting the required import permits, the actual price of purchase and the care and feeding of a Monkey probably exceeds the price of an analyst's consulatation services. After all, a monkey is far more valuable and useful than, say, that crazy Cramer dude on CNBC (seems like Ballmer's long lost brother doesn't he?).

It doesn't take a very sophisticated algorithm to outperform the average analyst. I suggest a far cheaper solution: My parents still have a lot of stuff from my childhood and I know that stowed somewhere in my basement is that Coleco ADAM. It has more than enough computing power to perform the task. What once came in handy for homework (it's BASIC was applesoft compatible yet the ADAM was much cheaper than an Apple) could be put to use as my investment advisor! I even have 5.25" floppies with SmartCALC spreadsheets of my Dad's portfolio as it stood in late 1986 I could use as a starting point!

Re:Why?

[bigberk]

Robotic stock analyst [perpetualbull.com] (disclaimer: for entertainment only, this is not investment advice ... some people email me and think these are REAL analyst statements). The point of this "joke" is to demonstrate who stupid most analyst statements are, they are just the same perpetually positive endorsement for stocks. Remember that analysts are employed by investment firms or banks and have a conflict of interest, they are overwhelmingly leaning in favour of telling you to buy stocks. Always.

Re:Already been done

[forgotten_my_nick]

I thought he didn't accomplish it and caused a market crash because of it?

3.14

Re:Existing algorithms

[Prof.Phreak]

Maybe there is a significant connection between keywords and stock movements that can be exploited by a software.

I've heard a talk on this. Don't remember where. The dude used spam-like classifier (keywords, bayes, etc.) for news articles, with `stockup/stockdown' [similar to spam/nospam] indicator (within a week of a news article).

Trained it on past articles (date published) and past stock data (yahoo). Then, just like spam filters, started classifying articles on whether they'll raise or lower a stock price for some corp.

The dude said he beat the market with that. Someone else commented: `it's been done'. Someone else said that `anyone can make money in a bull market' (he did his research in the last 3 years). I'd imagine there's truth on all sides---that articles really do correlate with stock price---and that the affect is too small to matter most of the time 'cause everyone's trying to exploit it (either manually or automatically).

Re:Brokers Are Crooks Or Just Used Car Salespeople

[ChrisMaple]

There are techniques available that average better than 20% a year: see www.aaii.com/stockscreens. There are also screens that regularly beat the major indexes. Though quite good, these numbers are not necessarily signs of fraud, nor are they the best that is possible.

There are several characteristics that, even if you take only one of them, will do better than the general market over time. One is high return on equity, which is good for several percent a year.

There's no necessity to be right 90% of the time. Many good systems are right for only 60% of trades and still beat the market significantly.

Re:Competing with other computers

[ChrisMaple]

Selling a huge number of shares does not necessarily depress the price if limit orders are used. Occasionally I've seen evidence of this happening, as a stock price bumps up against a price at high volume, stays at that price for an hour or so, retreats a little from that price and then rises to bump up against it again, and so on until the number of shares being sold is exhausted.

Re:What happened to investing?

[ChrisMaple]

401K money should go into broad indexes and healthy growing companies. The stability that strength and large numbers give overwhelms the effects of short-term traders.

Re:Verify quote

[Metasquares]

It isn't in the summary anymore, either. Strange.

Re:Economy question

[xenocide2]

Basically, investing isn't zero sum because someone is taking that money with the intent of creating more wealth than interest plus inflation. For example, issuing corporate bonds to build a new factory in Dayton, Ohio. If all goes as planned, the company makes their money back, pays off their creditors, and consumers are happy to have a new product to purchase. What obscures all this basic workings is risk. There's tons of risks in business: Your suppliers may corner the market and raise prices, your consumers may hate your design, your government might tax your product, your competitors might beat you to market. If I own stock and the company pays a dividend, it came from consumers buying the company's goods and services.

Investment markets like the NYSE and the Chicago board of trade arise from the desire to mitigate risk on investments. Rather than buying a lot of one company's bonds or stocks, buy several. You increase the potential that some of your bonds may fail, but hopefully decrease the chances that they all fail. There's tons more ideas like this out there, but I hope you get the idea. Trading involves two investors adjusting their holdings to match their risk preferences. Some people, especially young people, may prefer riskier investments, because they have a long time left to save for and can outwait a 10 year slump in the market. There does exist a class of traders that buy and sell stock looking to make a profit off of other investors, but generally profits come from the creation of wealth, ie firms paying people to make something and selling it to the public.