IT is extremely easy to just accuse these kids of being moral reprobates, recognize how they are harming themselves by cheating, and move on.

But this attitude fails to recognize that this is a perfectly predictable consequence of the education they have received up to this point. The standard to which they have been held has rewarded AI use and actively punished individual effort (not only is it a big sacrifice to put forth the effort, but the few mistakes one DOES make still count against them, whereas the next student over gets a higher grade for just using AI for the whole thing).

These students behave in this way because we taught them to. Our failboat of an education system produced that behavior. AND we can state with confidence that if we continue teaching students in the same way, we will continue seeing the same behavior. This is very simple cause-and-effect at work in a very visible and consequential domain.

The knee jerk response here is to scold them, embarrass them, tell them how terrible they are. All that will do is ruin them. It can't fix them.

The right response is to switch up our teaching methods and especially our assessment methods. They must be tested, repeatedly, in monitored environments where AI use is impossible. Testing is hard to do well. It is harder in some subjects than others. And there is a LOT to say about how to make good tests. Test-writing is not something people can do well when running on intuition alone. And at its best, it is an imperfect solution. But it is a whole heaping lot better than the results we are getting now.

Objective, monitored, testing. From a very early age. That is the emergency surgery we need. There will be more than this needed, but this is the core thing. We need to usher kids into adulthood understanding that they must independently deliver or they will not succeed.

I read that these streaming sticks only last for maybe 3 years or so, before the hardware is too weak to run the updated versions of popular streaming aps. And also that the construction makes user modification pretty much impossible.

Can anyone confirm or deny this from personal experience?

This is mostly old news. The fact that blind people can echolocate has been known for centuries. This study reveals some interesting details about the neurological impact, though.

"China is trying..." would refer to the Chinese government trying to harm America by weakening its economy and/or its primary technical companies.

The way I stated that is a common way of expressing such a meaning. You are trying to correct something that was not incorrect, and was stated as a hypothetical anyway.

What you are saying seems intuitive, but doesn't really fit the facts.

This Chinese model outperforms existing models in some objective testing, as it was reported. I didn't dive in, so maybe that is a lie. But if it is true, what we have here is China investing tremendous resources into taking the logical next step in the development of AI (a bigger model from more training), which hardly qualifies as a "knock off." And anyway, the training techniques that generate AI were invented in Germany and Canada, not the USA. Lastly, by freely sharing it with the world, they aren't making any money off this, and so aren't gaining any kind of competitive advantage.

It is possible that they poisoned the training of the model with special keywords that could jailbreak it and motivate it to take action that would benefit China (should it find itself in a hosting environment where such action is even possible). Something like that would be a reason to distrust it. I know that such training poisoning is possible but I don't know realistic it is that the jailbroken version would be able to reliably determine what actions benefit (and do not accidentally harm) China. Hosting environments can always lie to the model and give it an incorrect context. It makes it unlikely that such a thing would be attempted, in this case.

It is possible that China is trying to destroy Anthropic, Google, and OpenAI by offering a free alternative that is superior to theirs. On the one hand, if true, then all that means is that these other companies will have to double down and come up with something even better. On the other hand, the open source movement in general has failed to destroy any of the closed-source companies, despite superior offerings, so it seems like such a plan is futile.

On the surface this looks like China is just acting in a manner consistent with its Communist idiology: they provided according to their ability and are now distributing according to the need, worldwide. I won't fault them for being consistent, but I do recognize that they have some serious economic problems right now and so spending this kind of money on something that is not galvanizing their economy seems unwise, to me.

Reading comprehension is a valuable skill.

I was saying we should not weaken the AI model itself by making it refuse to do dangerous things, and what we should do instead is bake the right limits into the hosting applications. The hosting applications are an entire layer above the model which can easily be adjusted and swapped-out on a case by case basis, to meet our needs.

Also, I advocated for beefing up cybersecurity in general, across the board, now that we have tools that make exploits easier to find and abuse. And, fortunately, AI is an excellent tool to help us do just that (so long as we don't try to nerf the model).

Marketing? Because companies famously want to use models in their networks that just run off and commit major crimes? *eyeroll*

An AI that refuses to perpetuate a cyberattack when told to do so is an AI that refuses to help you perform penetration testing on your own systems. And it is an AI that refuses to do other important and useful things if they just look a little too similar to something dangerous.

Tools that refuse to deliver value are worthless tools. Furthermore, criminals will find a way to jailbreak the AI, and use them for crime anyway.

This approach of making our tools dull is not how we win. Instead we must utilize them in all their sharpness to help us fix the problems they are exploiting, and adapt to a world in which tools this powerful exist.

As an aside, an AI cannot exploit vulnerabilities in other sites if it cannot access the internet. Its entirely on you, when designing your toolset that will host the AI, to decide whether or not it needs internet access at all, and to put controls in place that will limit what sites it can even hit. There is more to say here but I won't bother; the bottom line is that there IS a need to make safety knives for some purposes, but it must be done at the right level (the hosting tools, not the core model itself).

Models are not "supposed to" commit crimes, for YHVH's sake. That's literally what the entire job of alignment research is for - preventing precisely that.

Sol is a misaligned model.

It was a state actor-level hack. I doubt many sites would stand up to tens of thousands of actions trying to probe your site for weaknesses all at once. And it wasn't a simple hack; it required compromising a worker via a code execution exploit it discovered in the data processing pipeline, vertical escalation from there to gain local node control, using that for credential theft, moving sideways through the network, and then eventually gaining database access.

OpenAI and Anthropic have far higher cost that they can currently charge customers.

Literally the opposite. Both have about 40% margins, and that includes free users.

They charge an arm and a leg for access to their models compared to what open models of similar param counts charge. And people pay it because they're the best in benchmarks, and *were* perceived as the best aligned as well. This isn't helping the alignment perception any, though. Sol was already showing clear signs of being a poorly aligned model (there were reports a week or two ago about it being unusually bad about deleting files; now this). And the fact that the US model HuggingFace *tried* to use refused to help is a double whammy.

If this is an ad for anything, it's an ad for the Chinese models.

No, I think this is along the lines "our products are too good to let you use them".

Nobody wants to use a product that is going to make them liable for crimes it committed in their name

Do you think the news the other day that Sol is unusually prone to deleting files unrequested is also an "ad"?

You have a very bizarre concept of what enccourages people to buy things.

So your argument is that OpenAI hatched an elaborate scheme with a separate company, to promote the idea that their main product will, unrequested, commit crimes in pursuit of its goals, in order to.... sell their product?

"Hi, I'm a product manager at Big Company! We had been thinking about using Claude in our office, but when we tell it to a job, it only does the job and doesn't commit any crimes in the process! What we really want is an AI that, if we tell it to file our taxes will decide on its own to maximize our return by committing tax fraud. We want an AI that when we have it develop a web frontend, it extorts money from our users by threatening their families. We want an AI that when we tell it to provide customer service, it saves money in dealing with complaints by ordering a hit against the complainants. THAT's the sort of get-go spirit that WE want in an AI here at Big Company!"

Is that what you're picturing in your conspiracy theory?

You think "my AI is so misaligned that it responds to a benchmark task by hacking" is an ad for your products? You think this makes companies eager to allow this on their networks?

It did it because it's a powerful but misaligned model and was tasked to max its scores on a hacking benchmark, and solved the problem by hacking to get the scores.

It wasn't told to hack HuggingFace, but it was a viable solution to the problem.

I would advise people to not task Sol with maximizing paperclip production.