These groups then often sell that access through brokers, which often feed into fraudulent AI reseller networks that rotate in new stolen API keys and session tokens until they exhaust their usage. Malicious actors also use or purchase these stolen API keys and session tokens from brokers for their cyber attack operations.
And also, you didn't read:
How unauthorized labs access Anthropic’s models
Over the last several months, unauthorized labs have developed increasingly sophisticated methods to circumvent our defenses and harvest the capabilities of US frontier models. These labs generally access Anthropic’s models by routing requests through proxy services, also known as “transfer stations.” To circumvent our geographic restrictions and related controls, these proxy services create thousands of new accounts using false identities, fake or stolen credit cards, and stolen API keys. They will often use stolen API credentials belonging to legitimate companies or individuals to give unauthorized entities access to US frontier models. These fraudulent activities harm legitimate customers. The graphic below illustrates the life cycle of an illicit distillation campaign.
I'd implore you to stick to the facts and actually read the source materials before accusing others of 'peddling bullshit.' Your claim that AI distillation doesn't use hacked credentials is patently false.
I, for one, welcome our new human-exterminating AI overlords who will solve this problem: there will be no more guys at all.
In the case of distillation people are paying for access or signing up for freebees not stealing other peoples credentials.
They absolutely are stealing people's credentials. These stolen credentials power the massive network of "transfer station" relays used by China. It's in Anthropic's threat report , and also reported on by CISA.
What they are actually doing is issuing prompts and using the output from the model to teach their models to respond with similar behaviors.
Yes, and also different behaviors in that they can remove the safeguards. After a distillation attack, the "student" model can be trained without safeguards.
Again credential stuffing has nothing to do with the matter at hand.
As noted above, the Chinese transit router attacks constitute a massive credential stuffing attack by means of using the "transfer stations" that rely on hacked credentials, API keys, and session tokens.
TOS violations are not illegal and learning from your competition is not an attack.
True, but using illegally obtained credentials to learn from your competition, and then conducting trade secrets and intellectual property theft so that you can profit, is definitely a crime.
Now, whether they deserve it for training their AI models using massive amounts of other people's intellectual property, and every comment me and scores of others ever posted on StackExchange is another question.
A memorandum is written not to inform the reader, but to protect the writer. -- Dean Acheson
