Lossy compression requires a utility function. That resides in the decision theoretic framework (engineering) not the prediction framework (science). The Hutter Prize is about the latter (Algorithmic Information Theory).
The contest went big early 2020, but so far no-one was able to beat the baseline. The discount has been introduced to ease participation and guarantee eventually at least one winner. The discount is around 1.5% per year, so should allow a first winner within a year, or at most two. The reason for the incremental discount is to prevent a quick win by whoever notices the discount first.
Disparaging the terms "master" and "slave" is hurtful to members of the D&S community. Is this the new idea of "inclusion"?
retchdog writes:
If BERT can magically sneak in some spooky black magic, then you're admitting that enwik9 is not an adequate data set for testing, end of story.
No. All BERT must do is beat the Large Text Compression Benchmark -- a benchmark that uses enwik9 as the corpus.
Megol asks:
But let's assume the information is important, why should the raw text be repeated instead of the myriad variants that express the same thing in a different way?
For the same reason scientists don't throw out the raw measurements in their experiments just because it departs from theory. One can assert that the form of the expression is irrelevant to the underlying knowledge, but this is a sub-theory based the agent's comprehension of the entire corpus. This becomes particularly relevant when attempting to assign latent identities to sources of information so as to model their bias.
Separating the 2 questions, 1) "Why not just use BERT?" and 2) "Why exclude parallel hardware?"
1) BERT's algorithmic opacity leaves open an enormous "argument surface" regarding "algorithmic bias". There are >100,000 hits for "algorithmic bias" due to inadequately principled model selection criteria used by the ML community, thence google's algorithms. Algorithmic Information Theory cuts down the argument surface to its mathematical definition of natural science:
Solomonoff Induction. Basically, it is provable that if classical (Cartesian) causality holds, distilling a dataset of observations to its Kolmogorov Complexity means you've factored out all the bias you possibly can if you claim to be "data-driven" in your policies. The BERT folks have yet to present a compressed enwik8 (100M) or enwik9 (1G) to demonstrate it is a superior unbiased model. They should do so. They don't have to submit an entry to the Hutter Prize for this, so they are free to use Google's entire TPU farm, or whatever. Let this be a challenge to them if they are serious about dealing with accusations of bias in their algorithms.
2) If one reads the relevant Hutter Prize FAQ answer, it becomes apparent that even a 2080Ti (or other GPU/TPU) may not be affordable to some of the best minds on the planet.
Lossy Compression Is Confirmation Bias.
One Man's Noise Is Another Man's Ciphertext.
Many if not most of the responses here posit that the degree requirement, even if not directly related to the job, is a cheap, if crude, way to filter out a deluge of job applicants.
If there is such a shortage of STEM workers that it is necessary to import so many that Silicon Valley has become majority Asian in less than a generation, it is rather difficult to justify such crude measures.
In reality, what is going on is that capturing positive network externalities has increasingly been the VC business model -- not invention. This creates monopoly profits that insulates management from bad hiring decisions. Rather than letting those bad decisions go to waste, Asian cultures, which can smell economic rent 10,000 miles away, ramped up their diploma mills (a diploma being the equivalent of a taxi cab medallion in terms of rent seeking), targeted the network effect monopolies, targeted the hiring authority of those companies, imported their "degreed" coethnics in huge numbers under the H-1b program, and focused more and more of the VC world on the rent-seeking network effect business model. The "guest workers" are then on a green card track which, when obtained, raises their value in the dowry market by tens of thousands of dollars. Everyone wins, except Western civilization and the folks that built it.
“Sandra Wachter, a researcher in data ethics and algorithms at the University of Oxford, said: “The world is biased, the historical data is biased, hence it is not surprising that we receive biased results.””
The single most subversive thing that can be done in the present environment is to financially back lossless compression prizes. One such prize is the Hutter Prize for Lossless Compression of Human Knowledge — although it needs to be expanded to include all of Wikipedia. Perhaps a more immediate prize would be based on compressing a wide variety of social science data. Sandy can then show everyone how smart she is by modeling the “bias in the data” so as to better predict it — which is exactly why compression is _the_ unbiased universal algorithmic IQ test.
...that if only the US hadn't stolen all those geniuses from the developing world, it would have developed so much faster?
Or, alternatively, that if only the US hadn't imported all those fake "geniuses" that the US would have already colonized the solar system?
It's over for Western civilization.
If you have a domain name under which you have a lot of content -- an example is kuro5hin.org -- and, after a decade or so you find yourself impoverished and stressed to the point that you can't renew the domain registration (as did Rusty Foster), a domain squatter jumps on it and holds it hostage for thousands of dollars. When that happens, frequently even "The Wayback Machine" is told to deep-six the archived content by the simple expedient of placing a robots.txt file in the home directory of the hijacked domain. "The Wayback Machine" then dutifully removes public access to the content. OH but the fun doesn't stop there! So now let's say you fork over the ransom money to the domain squatter, get the domain name back and remove the robots.txt. Of course "The Wayback Machine" then restores public access to all those articles... right?
WRONG!
archive.org does keep it stored and it is accessible to those with insider status, but no more public access EVER.
There really is value in hoarding history and if you can get away with it by doing it "on accident" all the better!
Polymer physicists are into chains.
