I think what they are, in effect at least, trying to say is that you do own (not license) the model output, but as a pre-condition for using their service you agree not to use the output to compete against them.
It's basically as if Microsoft said you can't buy our compiler if you are going to use it to build a compiler (or a clippy, or anything else that we do).
How they work, and/or how they are trained?
Anything specifically?
Distillation is the word used to refer to the practice of using the outputs of one LLM as inputs to train another, often smaller one.
The big US companies like OpenAI and Anthropic have usage "terms of service" that forbid you from doing this, but Gary Tan is saying this is unreasonable and you should be allowed to use the outputs of an LLM in any way you choose.
Specifically, Tan is hoping that, if allowed to, some US companies will choose to do this - train their own models with the help of outputs generated from these large/expensive OpenAI/etc ones, then release these new models in "open weights" form (i.e. downloadable, so you can run it on your own computer if you want to).
The "open weights" business model is interesting. The companies doing this can still make money by making larger customers pay, or by selling services (e.g. model customization), just as a Linux vendor can make money selling/providing open source software.
It does seem a bit weird.
When you pay for API usage, are you buying the model's output or are you licensing it under some restricted usage terms?
It seems what companies like Anthropic are saying is "you are buying the output, but we won't sell it to you unless you agree not to use it to compete with us".
It'd be like Microsoft saying we'll only sell you a C compiler if you agree not to use it to write a compiler.
The headline is questioning what Gary Tan is advocating for, namely that:
1) The US frontier labs shouldn't be allowed to impose terms of service that restrict how users can use the model's outputs
AND
2) Some US labs should take advantage of 1) to distill a, presumably smaller, model from the outputs of these frontier ones, and should release the resulting model in open weights form
I don't think they lose money on everything - it seems their coding subscription plans are probably not proifitable when maxxed out, as many people do, but at current full API rates it is likely quite a profitable business. It's not so hard to figure out, since the cost to provide the service is basically the cost of the hardware amortized over it's useful lifetime.
It does seem that AI is basically a commodity, and when cheap is good enough then the cheapest providers win, just as happened to PCs. No need to pay for a Ferrari when all you need/want is a Ford.
Yes, the AI companies, especially OpenAI and mostly Anthropic, are lobbying for government regulation, partly to exclude Chinese and open weights competition.
It is complicated though.
The guy, Jacob Coxon, you are presumably referring to, may well be regarded as part of Anthropic's own fear-mongering campaign and push for at least partly, maybe largely, self-serving regulation, but at the same time there are many developers working for these companies who say that the internal fear of what they are developing is almost universal.... and yet they keep working on it for whatever reason(s), whether that is pending IPO-riches or some justifcation about needing to develop AI to keep up with an arms race (which they seem to be the ones accelerating).
It seems what we need to do, and perhaps actually will, hopefully before it's too late, is to distinguish AI that is useful for everyday use like coding, and AI that is ridiculously overpowered for things like this and anyways dangerous, that indeed should be government regulated.
It's not an either/or matter of regulate or not - there needs to be some intelligence put into what is regulated or not, just as we allow people to own guns, but not fully automatic assault weapons or rocket launchers, allow people to drive cars, but do not license top fuel dragsters to be used on public roads.
> Controlling superintelligent AI is a fantasy.
Perhaps, but what we have today is something much dumber, yet arguably equally dangerous, in the form of these RL-trained LLMs.
It's like trying to beat DeepBlue or Stockfish as chess - it may not be intelligent, but by automating something and applying brute force compute to it, it becomes highly capable.
Many of these RL-trained LLMs have been specifically trained to be good at hacking and exploit generation, and also just for finding software bugs, which is only one step-removed. Take a hacking-trained LLM and apply brute force compute to it and you get the hacking version of Stockfish - you can call it dumb if you like, but it'll be able hack into many/most of the systems that have a vulnerability to exploit.
Could we at least control these hacking experts? Perhaps, or minimally we need their release to be slow enough to give time for all the potentially vulnerable systems (esp. infrastructure - air control systems, power/water plant control systems, etc, etc) to be checked and hardened against attacks.
Yes, for anyone interested a lot of the details are here:
Estimated around $20M at API prices.
Just to scoop a human researcher's work, then threaten to "ruin his career".
He surprisingly knowledgeable about AI, and started an AI-based film studio and sold it for $500M.
An LLM is just a stack of Transformer layers - they are great at what they do - prediction - but that is all they are built to do.
Humans are not just intelligent predictors. As a social species we have also evolved to (mostly) peacefully co-exist and co-operate, and a lot of our intelligence seems in fact to be collective intelligence.
If we want LLMs to behave in more human ways, then we need to make them more brain like, and start adding all the extra moving parts that evolution has equipped ourselves with.
Yeah - makes no sense to me either, but notable that this "you'll all be on UBI (government welfare)" is the best outcome he can think of.
Dario Amodei (Anthropic CEO)'s "Machines of Loving Grace" was the first of these, and if effect explains why others like Altman and Zuckerberg are chiming in.
Amodei's prediction:
1) AI will take ALL the jobs
2) Since people won't have any jobs, the government will have to provide UBI (universal basic income), aka food stamps
This is the most positive spin Amodei can put on the future that he is rushing to create, and not surprisingly people are not happy. The public increasingly see AI as an unwelcome technology that will create more problems than positive benefit.
This is at least part, a large part, of the reason that others like Altman and Zuckerberg are coming out with their own spin on how wonderful the AI future will be - because they realize public perception is massively against them (and also they are a bunch of arrogant pricks who think they are creating god, and are entitled to tell us how grateful we should be for it).
Well, I guess you could call a cruise missile a "mobile" application.
Bottom line it's a Russian weapons system enabled by an American electronics component.
> As for the Fermi's Paradox, I think the Occam's answer is Adam's Observation: "space is big. Really, really big..."
Right, and and given a failure to find the evidence we're looking for in a given neighborhood (~50 light year radius perhaps - SETI 50 years old), there are multiple ways to interpret this, including:
1) There is a) no life where we've been looking, and so b) we assume (why??) there is none in the almost infinitely bigger entire universe
2) There may be tons of life in this radius, just no advanced life
3) It seems more logical to assume that alien life does exist than not exist, so use this data to estimate an upper bound on how common it is
etc
We exist and have only been transmitting radio shows into space for 100 years or so, so put the shoe on the other foot and imagine an alien species in our own galaxy looking for life and not finding it. The Milky Way is 100,000 light years in diameter, and our measly "we're here" beacon has only spread out 100 years. Even if they are our neighbors only 50 light years away, are they looking for an ultra faint radio signal in Earths FM radio band ?!
It has apparently only fairly recently dawned on SETI researchers, or at least they are only starting to take it seriously, that maybe we are looking for the wrong signal in the wrong place.
The most scientific guess would have to be that primitive life - replicating metabolic soups - exists pretty much anywhere with the right conditions (it's just chemistry - why wouldn't it?!) - must be very common even within our own galaxy, even if it depends on a bit of luck (meteors hits delivering key elements). Complex multi-cellular life may be much rarer (on earth primitive life appears almost immediately, multi-cellular takes another 2B years), and complex life evolving human-level intelligence much rarer still (basically didn't exist for all of earth's 4.5B yr existence until "yesterday").
But, yeah, the visible universe is big enough, maybe ~1T galaxies, and then there is the non-visible universe beyond which is unimaginably bigger (maybe 10^20 times as big, at minimum).
Your mode of life will be changed to EBCDIC.
