Forgot your password?
Close
typodupeerror

Comment How much safety does this give? (Score 1) 1

by jd (#66266560) Attached to: How to improve C/C++ code quality, random thoughts

It obviously gives us guarantees in terms of what the source will do and what the compiled code should need.

These do not provide memory safety, so memory bugs would not be solved by these. They will find unexpected behavioural issues, though.

The precondition and postcondition statements are not cunningly-disguised assert statements because they're not asserted at runtime. They're enforced at static check and/or compile time.

The policy hints mean that if the kernel is doing something you're not expecting, you find this when testing the code. Equally, though, it means that users cannot cause the software to venture into realms unknown through many of the usual attack vectors because those will require permissions that aren't there.
User Journal

Journal Journal: How to improve C/C++ code quality, random thoughts 1

Journal by jd

I am thinking about the following concept. Take doxygen comments and extend them as follows.

1. Permit identification of pre-conditions for functions (what has to be true when the function is called)
2. Permit identification of post-conditions for functions (what is intended to be true when the function exits)
3. Permit identification of hints about what kernel operations are being used by that function

Submission + - A fundamental flaw leaves LLMs strikingly vulnerable to attack (technologyreview.com)

Submitted by joshuark
joshuark writes: It is impossible to make large language models fully secure against hacks because of a fundamental flaw in how they work, a team of researchers argue in a paper presented at the International Conference on Machine Learning, a top AI conference, this month. The claim has huge implications for the safety of this technology.

By taking advantage of this flaw, which concerns how LLMs identify who or what is giving them instructions, the researchers were able to make popular LLMs spit out information they had been trained not to provide, such as how to synthesize cocaine and how to sabotage a commercial aircraft’s navigation system.

“There’s a real probability that this is going to be a problem that’s fundamentally unsolvable,” says Charles Ye, an independent researcher and coauthor of the ICML paper.

Companies will typically hire teams of human testers to try to come up with novel attacks that break existing guardrails, a process known as red-teaming. Model makers also use LLM super-hackers (such as OpenAI’s GPT-Red) that find and exploit weaknesses in other models to automate parts of this process. The goal is then to take those attacks and train a new model to resist them and anything that looks like them.

The problem, says Jasmine Cui, another independent researcher and coauthor of the paper, is that the approach amounts to giving the models a list of things they shouldn’t do. But no list is exhaustive. “It’s like watching The Simpsons and they have Bart writing ‘I will not say something inappropriate to my teacher’ a hundred times,” she says. “And he still does things that are pretty crass anyway.”

The ICML paper describes attacks against several of OpenAI’s models, but Cui and Ye say that they have since seen similar results with models made by Anthropic, Alibaba, and DeepSeek.

Cui and her colleagues wanted to find out why an attack like chain-of-thought forgery was so effective. They suspected it had something to do with the mechanism that LLMs use to keep track of where their instructions are coming from.

But what Cui and her colleagues discovered is that LLMs are in fact very bad at keeping track of different roles. In a series of experiments that looked at what was going on inside a handful of different models, the researchers found that LLMs seem to identify the role of a specific chunk of text not by the tags around it but by the style of that text and the words it contains.

The upshot, the researchers claim, is that all an attacker needs to do to hack an LLM is write text that spoofs a certain role. And because roles are a fundamental part of how LLMs work, no amount of training will fully solve the problem.

Ye is worried that nobody is ready for what’s coming. “There’s going to be a huge economic incentive for people to do jailbreaks and prompt injections,” he says. The best defense could be to expect the worst. Organizations shouldn’t trust LLMs, and they should expect that anything done by agents could be unsafe, he says: “That’s not a great solution, but it just might be what we have to do.”

“It’s really incredible that these things are being deployed everywhere to control super-critical systems,” he adds. “There’s been no study of the fundamental science here. We’re all doing it ad hoc.”

Submission + - AI Companies Destroying Books At Scale (futurism.com)

Submitted by nightflameauto
nightflameauto writes: AI companies are purchasing books from pre-AI times. The claim is that those books are free from the defects of AI generated text, and therefore more valuable for ingestion into AI datasets. In order to accomplish this, they are tearing the books down to be scanned, then destroying them as the scans are completed. This includes rare and out-of-print books that may be some of the few copies of any given published work left in existence.

Submission + - Is Mark Zuckerberg Actually TRYING to Destroy Meta? (futurism.com)

Submitted by fjo3
fjo3 writes: Lately, it almost feels like Zuckerberg is trying to destroy his own company. He’s burning through mountains of cash in a desperate attempt to keep up in the AI race. Yet despite the untold billions it’s spent so far, it’s being destroyed by OpenAI and Anthropic. Its employees have even resorted to using AI models made by its competitors, a humiliating reality check for how far behind Zuckerberg’s efforts have fallen.

It’s also a financial nightmare. Capital expenditures have risen dramatically, erasing any appetite for Meta on Wall Street. Its stock price “has been dead money for more than a year,” as Yahoo Finance notes, as investors continue to debate whether Zuckerberg is chasing the AI industry as it careens off a cliff — or edging ever closer to an AI-fueled industrial revolution.

Submission + - The first active shooter suppression service

Submitted by Anonymous Coward
An anonymous reader writes: Schools to tackle active shooters with pepper-spraying drones that can ram attackers

Pepper-spraying drones aimed at tackling active school shooters are being rolled out at campuses across three states.

The drones, part of the Campus Guardian Angel program, can ram into targets at speeds of up to 70 mph and are capable of weaving through the air to avoid gunfire. Three schools in Florida, five in Georgia and one in Colorado are part of pilot schemes using the aerial technology.

Submission + - Your Brain Can Rewire Itself to Allow True Multitasking (sciencealert.com)

Submitted by alternative_right
alternative_right writes: Our daily lives are built on multitasking, but are our brains actually doing two things at once, or just switching very quickly between them?

Well, it depends. The science shows it's the latter, for cognitively demanding work.

But in a new study published in the Journal of Cognitive Neuroscience, researchers from Georgetown University Medical Center in the US have revealed that we can put certain tasks on autopilot in a way that enables something closer to true multitasking.

Driving is the perfect example: When you take your test, your entire mental and physical energy is concentrated on executing the right combinations of movements and thoughts.

After a decade behind the wheel, the brain is no longer consciously thinking everything through in great detail.

Comment Re:Does this end them sooner, or is it irrelevant? (Score 1) 38

by jd (#66256222) Attached to: Anthropic's New Opus 5 Model Rivals Fable 5 For Half the Price

I mostly do heavy multi-disciplinary engineering problems (these I also use for checking an AI, as they're typically not good at these sorts of problems), complex coding, OS analysis, stuff like that. However, sometimes I do throw the occasional odd-ball - I've used ChatGPT to propose a workable quantum mechanics that will cope with Doctor Who canon, for example, and to produce an outline for a story in which symphonic metal appears in 1964 that is compliant with current sociological and psychological models of behaviour.

Claude Opus 4.6 was coping surprisingly well with just about everything I threw at it (but ran out of credits fast), but Opus 5 is churning out incoherent babblings to the point I'm worried I may have accidentally summoned Cthulhu.

But Gemini, Grok, and DeepSeek got hopelessly confused on just about everything past a very low level of complexity. They can handle large problems, yes - Gemini has a huge context window - but complex interactions baffle them.

ChatGPT is able to identify issues correctly, but can only outline solutions, it's just not good at depth. 5.6 is a lot better, but still not good at deep answers. ChatGPT is also prone to agreeing for the sake of it, which makes me nervous about trustworthiness.

Comment Re:Does this end them sooner, or is it irrelevant? (Score 1) 38

by jd (#66256078) Attached to: Anthropic's New Opus 5 Model Rivals Fable 5 For Half the Price

That is fair enough. I've been trying out Kimi on the free model, and have subscriptions to Claude and ChatGPT. If Kimi is actually as good or better than ChatGPT, at the pro level, then it might be worth my while moving over as ChatGPT has become very disk-hungry of late and I'm pushing right to the very limits on what it can reliably process.

Comment Echolocation is a fascinating skill (Score 4, Interesting) 40

That humans have sufficiently directional hearing is perhaps the most impressive part. Once you have that, then the rest really just follows,

However, this continues the unexpected senses in humans, one of the first discovered was that humans have a weak magnetic sense.

The potential, both in fact (if you can "see" walls and gaps by clicking then you can presumably navigate a cave even if your light source goes out) and in fiction (this one should really be obvious), is considerable.

Slashdot Top Deals

How come financial advisors never seem to be as wealthy as they claim they'll make you?

Close