LLMs have two effects here:

1. They help the white hats find and fix bugs
2. They help the black hats find and exploit bugs

These two effects cancel each other out to some extent (depending on which "team" is more proactive in its bug-discovery and consequent activities).

Eventually all of the low-hanging-fruit will be picked, and the current deluge of AI-induced CVEs will be reduced to only those bugs that were subtle enough not to be noticeable by anyone except the latest-and-greatest AI models.

Most maddening for open source projects is the number of false positives exacerbated by multiple people trying to 'help' by running effectively the same security audit as a bunch of other people have done and trying to open issues that are duplicate...

Huge pain in the ass dealing with contributions from people who don't understand enough to analyze their LLMs "findings" and just pass them through "in case they are helpful".

Might be nice once the fad of "everyone contribute by running duplicate reviews" subsides though.

Note that this isn't even new to AI, the kernel has always been a flood of CVEs. The AI spotlight puts a bigger spotlight on it, but it has been a mess.

The real problem is that the average security team is mismanaged and forced to use terrible tools that report on yet fail to reconcile the status of the CVEs on behalf of the user and instead drives the team to have to figure out how to take care of it themselves.

Some twisted security mindset of going direct with CVEs as 'the most thorough vendor neutral approach' without instead leveraging distribution security advisories. So you end up with a security tool claiming you have thousands of security issues on an up-to-the-minute patched instance of the latest enterprise linux distribution because they backport and tools don't understand the version numbers.

In the industry, so many security teams mandating CVE management software that is absolutely stupid and pushes off all the hard bits to the user.

It sees 6.12.0-211.34.1.el10_2... Well... none of that makes sense so it just assumes that it's just plain 6.12.0 and demands the user reconcile the reality. Now the security vendor *could* maybe integrate with the major linux distributions advisories... But no, more CVEs is better, when you demo that your product finds hundreds of CVEs, that just proves it is being thorough as far as upper management is concerned.

They offer up sound guidance here, skip direct CVE management and lean on your distributor. If you are doing CVEs direct, at *least* use the patched files to cross-reference against your build to see if it even in theory could matter. However the problem is the business of the security industry doesn't prioritize those, and so we have stupid tools inflicted on people, and pushing back against those tools carries a stink of "must not care about security then".

Depends on the number of realistically 'false positives'.

I've known a few people who find the kernel CVEs particularly unreasonable as they tend to aggressively assume security implications. If they grant a CVE to a 'mere bug', no one is going to get too grumpy over that specific item. If someone believes they have a vulnerability and do not see a CVE, then people get riled up. So some feel the kernel is just granting CVEs to avoid pushing back.

The other headache is the monolithic nature of the project. "Linux" covers just everything. A potential security issue in a device never seen outside of PA-RISC systems 20 years ago? It's a "Linux" issue, so every x86 system will be flagged as 'affected' by security software that cannot deal with nuance

Of course, we are here mainly because the kernel team largely recognizes the practice of trying to apply only security updates while avoiding 'only bug' fixes as pretty insane. So err on the side of caution make CVEs extra unmanageable because realistically it was a pretty crappy strategy for such a complex project anyway.

Broadly speaking, CVEs are usually pretty bogus, but a small percentage are very real and critical issues. You can't use the 'score' to really measure this either, it's not very good in the first place, and for example I saw the exact same issue in a C library and a python binding for that C library, and for whatever reason they graded the C library as 'minor' and python binding for that library 'critical', despite the python binding being nothing but a ctypes wrapper around the c library...

Like ask Jeeves but with get 'er done attitude

Marketing? Because companies famously want to use models in their networks that just run off and commit major crimes? *eyeroll*

You expect very slop happy software developers to create effective safeguards?

Models are not "supposed to" commit crimes, for YHVH's sake. That's literally what the entire job of alignment research is for - preventing precisely that.

Sol is a misaligned model.

It was a state actor-level hack. I doubt many sites would stand up to tens of thousands of actions trying to probe your site for weaknesses all at once. And it wasn't a simple hack; it required compromising a worker via a code execution exploit it discovered in the data processing pipeline, vertical escalation from there to gain local node control, using that for credential theft, moving sideways through the network, and then eventually gaining database access.

OpenAI and Anthropic have far higher cost that they can currently charge customers.

Literally the opposite. Both have about 40% margins, and that includes free users.

They charge an arm and a leg for access to their models compared to what open models of similar param counts charge. And people pay it because they're the best in benchmarks, and *were* perceived as the best aligned as well. This isn't helping the alignment perception any, though. Sol was already showing clear signs of being a poorly aligned model (there were reports a week or two ago about it being unusually bad about deleting files; now this). And the fact that the US model HuggingFace *tried* to use refused to help is a double whammy.

If this is an ad for anything, it's an ad for the Chinese models.

No, I think this is along the lines "our products are too good to let you use them".

Nobody wants to use a product that is going to make them liable for crimes it committed in their name

Do you think the news the other day that Sol is unusually prone to deleting files unrequested is also an "ad"?

You have a very bizarre concept of what enccourages people to buy things.

So your argument is that OpenAI hatched an elaborate scheme with a separate company, to promote the idea that their main product will, unrequested, commit crimes in pursuit of its goals, in order to.... sell their product?

"Hi, I'm a product manager at Big Company! We had been thinking about using Claude in our office, but when we tell it to a job, it only does the job and doesn't commit any crimes in the process! What we really want is an AI that, if we tell it to file our taxes will decide on its own to maximize our return by committing tax fraud. We want an AI that when we have it develop a web frontend, it extorts money from our users by threatening their families. We want an AI that when we tell it to provide customer service, it saves money in dealing with complaints by ordering a hit against the complainants. THAT's the sort of get-go spirit that WE want in an AI here at Big Company!"

Is that what you're picturing in your conspiracy theory?

You think "my AI is so misaligned that it responds to a benchmark task by hacking" is an ad for your products? You think this makes companies eager to allow this on their networks?

It did it because it's a powerful but misaligned model and was tasked to max its scores on a hacking benchmark, and solved the problem by hacking to get the scores.

It wasn't told to hack HuggingFace, but it was a viable solution to the problem.

I would advise people to not task Sol with maximizing paperclip production.