Follow Slashdot blog updates by subscribing to our blog RSS feed

 



Forgot your password?
typodupeerror
For the out-of-band Slashdot experience (mostly headlines), follow us on Twitter, or Facebook. ×

Comment: Re:The problem with data driven science.. (Score 2) 55 55

I also see a trend that people look for correlations, find correlations and then draw some conclusion without any proof of causation. To me it strikes me most for economics. Public policy is set based on those correlations.

It is very counterintuÃtive but correlation research means nothing, especially in economics. Correlation research would be an amusing way to spend your time and get to know some variables, but correlation research is being used to inflence people. Repeat after me: correlation means nothing. If you find a correlation luck has hit you. Or luck has been manipulated to serve some point. Correlation means nothing whatsoever. Articles describing correlation are a waist of your time. You should not act based on correlation research.

Now if you take big datasets with lots of variables and you test correlations between those variables, you will find strong correlations. Correlation here, correlation there, correlation everywhere. If you do millions of tests en tweak your parameters, correlation is all yours.

But luckily now you know: correlation has no pratical use in your live.

Comment: Re:Of course it is a problem (Score 1) 408 408

Data mining is indeed a very mediocre scientific activity. Correlation on itself means nothing at all. If you want to proof something the correlation should be 100% and you should be able to explain why the correlation exists and replicate it in controlled experiments. The problem is that those slam dunk scientific discoveries are all or mostly allready found. And nowadays the poor scientists need to find something to bolster their path to glory.

Good science could be: find a correlation an proof the causality. But a lot of studies stop at the correlation. That's what fills newspapers nowadays. 'You get fat from diet coke since most people that drink diet coke are fat'.

Some scientist try to eliminate all other reasons and then decide that their causality is the only one that explains the correlation. But in effect they say: those things correlate and I 'the superintelligent scientist with multiple PhD's' cannot find another explanation and that is why my explanation must be true.

For background you should listen regularly to 'more or less: behind the stats' http://www.bbc.co.uk/programmes/b006qshd
You can listen to podcasts that are interesting and fun to listen to. And some of the older ones are absolutely great. They gave me great insight in the workings of media (and science).

The only downside is that if your girlfriend tells you something she heard on the radio and you answer her: correlation is not causality, she gets upset.

An optimist believes we live in the best world possible; a pessimist fears this is true.

Working...