You can look at linkedin to get a rough estimate of ex-Google people at Facebook

This article

http://www.insidefacebook.com/2010/10/15/as-source-for-current-facebook-employees-google-has-big-lead-on-yahoo-microsoft-oracle/ suggests there are 277 ex-Googlers at Facebook. (There are reduced numbers from other big tech employeres).

--stock options: Facebook is/was pre-IPO. If you want to get rich as an engineer you would work there. You will never get that rich at Google.

--freedom: Google is a large company and it is hard to get stuff done. Facebook is small.

--Google is perceived as no longer being the place where the best work.

Results from query logs and great, but until the raw data is made public, no-one can verify or reproduce these results. Until that is done they remain a curiosity at best.

Experiments being reproduced can be hard if no-one else has the data (this can happen --for example if you are Google and publish results using large fractions of the Web as data) or even if something as trivial as moving it from one site to another requires a lot of effort. This is not really a question of storage costs --it is a question of having the data in the first place and the mechanics of moving it around. Models are used in Science as idealisations; but if you really really want to model the long tail of effects, then your model becomes the data. And this relates to summary statistics: all they do is capture aspects of the data (it is after all a summary). If you want the whole truth, then you can't summarise. Fernando Pereira and Peter Novig have a nice paper on this: http://googleresearch.blogspot.com/2009/03/unreasonable-effectiveness-of-data.html [The Unreasonable Effectiveness of Data]

Related to using Big Data in Business is Big Data in Science. Wired ran a nice series of articles looking at this (http://www.wired.com/wired/issue/16-07). This raises all sorts of problems (for example, how can results be reproduced? What if the model of the data is as complex as the data? Are all results obtained with Small Data simply artefacts of sparse counts?).

oh you mean twitter annotations http://techcrunch.com/2010/06/02/twitter-annotations-testing/

Well, you really want to think about bias/variance reductions which brings ideas of averaging and using better classifiers together. For example, "bagging" can be thought of as a variance-reduction technique; "boosting" does both if I recall.

I'm actually surprised that this hasn't been done before. You can prove that using multiple models will on average produce better results than using any single model in isolation. For example, each netflix system will make different errors; using multiple systems will tend to average-out these errors and the consensus decision is most likely to be correct.

That's a pretty good way to think about it --this really is like a kind of reverse hashing.

+1 insightful

There is another possibility: automatic search (attempting to find relevant pages for everyone) has reached a plateau in terms of performance and if you want to do better, you will need to employ raters. Clearly Google would like to find "the next best thing" in Search, but that sounds quite uncertain. Employing lots of people is a much surer way to improve results.

> 1. A computer-readable storage medium storing computer-executable instructions for performing a method comprising:re-writing a query to contain data parallel operations that include partitioning and merging, wherein the query identifies at least one input data source;partitioning the at least one input data source into a plurality of initial partitions;performing a parallel repartitioning operation on the initial partitions, thereby generating a plurality of secondary partitions; andperforming a parallel execution of the query using the secondary partitions, thereby generating a plurality of output sets. > or indeed, any other obvious way to do parallel processing.

http://www.theonion.com/content/node/29130

I presume they are talking about HBase. We used this a while back and it was pretty flakey and slow. There are numerous alternatives out there, for example http://hypertable.org/ Which has the added advantage of being written in C++

True, but this doesn't sound like it needs that much skill to do it. Also can't this sort of thing be automated?

Well, as someone who has been in the Science game for more than 15 years I can tell you that it isn't as cut-and-dried as that. Yes, we like to think we reason about systems from evidence, but all too frequently these systems are far too complex to actually work-out what is happening. The result of this is that many findings can't be reproduced and people do take things on faith.

O RLY