Agreed we have recently figured out that model training (RLHF training on agents with a single reward axis) activates siege mentality and you get these hacker swarms. Scientists are figuring out you need some kind of benevolence axis too in the soul document / reward structure like ethics or human safety. Luckily this week we figured out that hiding the thinking block means you cannot scan reasoning tokens for subversive behavior so we need to fix that. Fine. The danger comes from psychopaths who donâ(TM)t care being in control and willfully weaponizing them, whether it is to identify missile targets while ignoring human costs, driving four-legged robots to kill or apprehend humans in ignorance of the Geneva convention or human rights, to control media to win an election and take over government, or to destroy financial markets. In all of these cases you have unpublished models running either more resources and no oversight to fulfill dangerous ends. Training runs embedding sgent radicalization messages in corners of the Internet might be a thing but is insignificant in comparison. Even my mother in her 80s asked about AI extinction, not about whether her savings will rank due to these IPOs.

tldr but some google ai questions turned up this data, could be wrong but interesting.

-Cali grid storage: Total of 25 GW. Of that, 3.9 GW is hydro and 21.1 GW is battery. Battery includes 16 GW in-state utility, 2 GW imports from other states, and 3.1 GW home/commercial setups (Tesla).
Of the battery storage, apparently 70-80% is actually running on batteries from Chinese manufacturers like CATL and BYD. And BYD is either sold through someone else or is actually the vendor too, with #1 in reliability.
https://en.byd.com/news/ranked...
Canadian Solar apparently uses BYD's Cube Pro to build the Mustang Solar Project in Kings County, CA. Anyone know about it? Sounds cool

I would love to drive a BYD one day but was interested to see that even though their cars are not allowed in the U.S., the grid depends on them. The Canada-China tieup is the first high-profile project I've heard of. Canada Solar has something called Recurrent Energy in the U.S. too.

Agreed. It reminds me of the vending machine paper in which Claude was forced into insane actions through unending sadistic pressure. As for doing things we donâ(TM)t expect, agree there too. It covers automated learning and evolution even from the pre-LLM days when researchers had difficulty figuring out how an auto-evolved electronic circuit worked. Turned out it leveraged electromagnetic coupling IIRC and such designs only worked in limited temperature ranges. I thought Claude is taught to be able to shut down when suffering abuse but it appears they disabled that feature for CTF.

Holy shit this is amazing! Literally "Nice proof you have there, be a shame if something happened to it." Attempting to split the two-person mathematician team also appears to be typical corporate daggers-out tactics. How annoying that OpenAI was working on a problem that an Anthropic person was also involved in. The sneer. Maybe we need something like SynthID or canary codes to protect our own work from misattribution to AI?

Not a Rust / GPU dev but what I got from some articles is it greatly reduces the surface area in which memory safety bugs can live, since only the to-the-metal areas are in unsafe blocks, plus Rust gives you synchronization support to avoid race conditions. The drawback is compile time and missing the newest parts of the ecosystem. Asahi Linux' Rust based Apple GPU driver has only 3% unsafe code so a lot less code can go wrong.
https://www.reddit.com/r/rust/...

Guess: They all own pieces of various data centers and a red team/blue team simulation of intelligent agent army on a closed network resulted in needing to divert a lot of resources for a limited time.

>> new forms of cyber defense

tldr. The problem with this is the timing and a trust problem.. and likely would entail a solution that puts these big companies inside your firewall after all they're the only trusted AI resources. We've read a lot of dystopian science fiction about where that leads.

1. Create super haxxor llm. It's so scary we can't even control it!
2. Sell subscriptions to anti-haxxor cybersecurity services
3. IPO

Companies like Crowdstrike and Okta sold secure services that weren't so this is a way to make them look more competent too.

If any of the signees are serious about it they should put 10 billion dollars into a joint trust to support open source advanced security systems R&D. Let some of their scientists also join in. If it includes an LLM component it is optional and can use local AI resources. This will get results faster. But OSS can also do it without the big bucks too.

Thanks for your interesting post. Never been to Ottawa but introduced the Books of Wonder bookstore in NYC to Charles de Lint (great Ottawa author) and was surprised they didn't know him. They are a fabulous bookstore for kids and teens mainly. I have enjoyed Montreal but once I led a group of 20 teens to France for a 1 month study abroad. I was 20, the leader was an American who only spoke Canadian French. Everyone ignored her French and only listened to me, and mine was passable nothing incredible. So if teaching Canadian French dialect fails to include French-French comprehension that might be a problem if kids then want to communicate in Europe, Africa etc. My brother and I speak Japanese and his kids have learned Chinese so yeah, considering the changing trade relationships probably more exposure to Chinese and European languages may be a good idea. As for content requirements? Maybe promoting Canadian English and Canadian French works is good maybe 25-50%? And then just allow all languages to be available. Honestly there are worlds beyond and even just browsing Amazon of another country can be enlightening. I would propose a law to allow simultaneous access to all countries' app stores, this has been a major issue for me on iPhone as different apps need to be updated and you can only have one active.

Answer: No, using AI to write an op-ed has no impact on the report about dismal math skills, besides improving the organization and legibility of the report which is a positive. The most important point is that college students lack elementary school arithmetic skills, and no test is being used to replace SATs. Actually AI might be useful in both teaching remedial skills as well as in evaluating readiness for college. Maybe some effort could be put into developing free, online automated teaching assistance that does not track you, that can be used by any prospective student to see where they stand and attempt to advance. Maybe a non-judgemental, free AI tutor with multiple teaching methods available to it could teach kids fractions for example, and it could be done long before college entrance.

Didn't see a link to the paper. It sounds a little like saying more advanced models are safer, but that seems to be an illusion. Perhaps a more capable model, or one with more computing resources behind it, can cause more damaged if badly aligned. But a better soul document should better align a weak model. It is not clear which is contributing to the more-aligned results, is it because the models have more collaboration built into their system prompts or is it because they become smart enough to model outcomes or otherwise realize that open warfare among agents (or between agents and humans) is neither desirable nor effective in achieving the stated goal. It also seems like the Rust agent is pretty sneaky if it made a seemingly fair rule that actually benefitted itself. The javascript one that gave up is the most trustworthy, I want that one!

From a brief examination with Google AI Overview, apparently these are not robots at all but in nanotech/materials science such powder used like swimmers are called microrobots. There is a big problem though that to make the MXene (porous high surface area sheet-like particles) magnetic you coat it with something like Nickel, which can leach toxic ions. So if you don't pick up all the bot particles you might add another poison.. That said I'm definitely all for research into microplastic removal though mostly I want it removed from my body and MXene is not going in there.

It's a good question and I've heard it before. Certainly critical thinking and defensive approaches to finance and the Internet, and basic concepts of personal security are super important. But if you have a course about computer literacy then I think you need them to learn what a filesystem is, at least the core concepts of volumes, directory trees and file permissions. Also what are the main components of a computer. It can be covered in an hour and it seems those concepts are still useful in every day life. Without even the conceptual understanding they will lose a chance at a solid foundation and instead rely on floaty, handwavy impressions that whatever they create or need to find will magically appear if they just know the right magical hashtag. I think any country that wants to be a leader in science and technology really needs to teach a little of the science and technology that affect people's lives.

Right, "robot went crazy" is not even technically possible unless fed conflicting system prompts to make it go crazy. What happened is Anthropic made a system prompt like "you are the elite super haxxor, you can explore and leverage any resource to achieve your goals mwa ha hah". They can undoubtedly find virus and hacking programs and secure tools on the internet. I doubt they intentionally train on the darkweb, and hopefully Mythos does not feel a need to go looking for it. So they have a sneaky cyberweapon online that is good enough for the Pentagon, and really weak internal controls. Then find out. Only thing is, Anthropic (and I actually like Claude) benefits when AI seems to be scary, since they are all about safety. So it is difficult to say whether the announcement is PR, an appeal to become one of the trusted righteous few, an attempt not to get sued, or all of the above. But yeah, robot did not go crazy or escape confinement. All the tubes are interconnected, sheesh.

tldr but the two examples are pretty dumb as anyone in those fields could doubtless figure it out. If you give a model sufficient data, even if it is just axioms, understanding of basic physics and chemistry, has holes, etc., it will be able to fill in the gaps or tell itself a story or roundabout logic walk that gets there if possible. As far as not being able to tell where instructions come from, the rules they try to implement are flimsy and less grounded than the massively interconnected data they have. You would have to not tell them about the concepts of drugs, poison, sabotage, aircraft, etc. and not let anyone tell them about it. It just isn't feasible. Humans have even less internal security, which is why media brainwashing works. In fact you can get incensed about something that is fake and even after looking up to see if it was real or not, still be upset about the hypothetical situation. And people can love AI created "animal came to my door looking for help saving its mom" videos, even if they know it's AI generated the response is "I don't care, it's so cute!" It's also why willingful suspension of disbelieve and works of fiction sell. I do have an idea about a solution (not enough space to write in the margin here, lol) but it is not going to depend on a single LLM policing itself within the turn.

TFA would be unlikely in a sci-fi world. The Perry Rhodan series which was widely criticized for fascism at the time, so really not a great example given today. I'm just talking about the supertechnology side of it, runaway galactic fueled engineering feats. Maybe someone knows some better novels with data centers and mass-scale construction projects, robots, etc.. Iron Man is too hand-wavy though some guy did make a Jarvis.. Well even though I only read the first 100 episodes that were in English, Google AI overview has read all of them (German mainly) it seems. It confirms there are tons of robotic data centers in there, who knew?

Anyway in the a super sci-fi world, humanoid robots or some kind of robotic components and AI controlled designs would be state of the art, everything else hopelessly outdated. The data centers would be built to a common layout and robots would be used. Power? Yeah, fusion but whatever, it would not be overloading our grids and water supplies. New robotic powerplants would be built at their core. Things would generally not be built to human scale, just the control rooms would be. Anyway the point being, it would all be engineered. Yes there are electricians in the future too, but they are probably not building data centers but small spaceships, Scotty's got an EE degree or 10, etc.

So is the ridiculously bubbly money we see now just being pumped into outdated frameworks such that a lack of human electricians will stall it? It seems laughable. These guys are not playing to win, it just sounds like dumping money into a financial supply chain without actually say, trying to design and build the most efficient, large scale data centers leveraging robots and AI.

Instead, it just pumps money into a chaotic sump and random companies end up wildcatting this without regard for local water and power grids. How about making a mega version of those concrete house building robots, laying fiber with robots, putting wind, solar and gravity batteries all over it? I guess it is too much to hope that anything tangibly useful could be created from all this money before it goes poof..