Comment Re: We are going so fast we need to slow down! (Score 1) 98
In the case of distillation people are paying for access or signing up for freebees not stealing other peoples credentials.
They absolutely are stealing people's credentials. These stolen credentials power the massive network of "transfer station" relays used by China. It's in Anthropic's threat report , and also reported on by CISA.
What they are actually doing is issuing prompts and using the output from the model to teach their models to respond with similar behaviors.
Yes, and also different behaviors in that they can remove the safeguards. After a distillation attack, the "student" model can be trained without safeguards.
Again credential stuffing has nothing to do with the matter at hand.
As noted above, the Chinese transit router attacks constitute a massive credential stuffing attack by means of using the "transfer stations" that rely on hacked credentials, API keys, and session tokens.
TOS violations are not illegal and learning from your competition is not an attack.
True, but using illegally obtained credentials to learn from your competition, and then conducting trade secrets and intellectual property theft so that you can profit, is definitely a crime.
Now, whether they deserve it for training their AI models using massive amounts of other people's intellectual property, and every comment me and scores of others ever posted on StackExchange is another question.