Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 



Forgot your password?
typodupeerror

Submission + - AI slop? Not this time. Generative AI found 50 real bugs in cURL — and cou (etn.se)

jantangring writes: AI-generated bug reports are usually trash. But when a security researcher used LLM-based scanners the right way, he found 50 real bugs in libcURL. Swedish tech journal talks to cURL maintainer Daniel Stenberg and to Joshua Rogers, the Australian hacker / security researcher that is using AI tools to uncover loads of old bugs in open source projects.
Generative AI has now proven that it can independently discover new vulnerabilities in high-quality source code. New generative AI tools are suddenly digging up bugs that traditional static analysis tools have been overlooking for years.
”I’m actually overwhelmed by the quality of some of these findings”, says Daniel Stenberg, maintainer of the file-transfer library cURL, in an interview with Swedish industrial electronics news publisher Elektroniktidningen (etn.se).
In a well-known talk this August, Daniel Stenberg warned that he and his team were being flooded with AI-generated bug reports — wrong, confused, hallucinatory garbage created by generative AI.
Such “AI slop” has begun to waste valuable time for open-source maintainers, not only in cURL. The community is struggling with how to stem the tide.
Still, banning AI wasn’t the solution, Stenberg argued back then. He believed that AI might yet prove useful.
And he turned out to be right. In September, a batch of cURL bug reports arrived that has so far led to 50 fixes in the cURL library source code.
It marks a clean break from the previous wave of junk reports. There may have been the odd valid AI-based bug report before, but this time, Stenberg’s team implemented fifty fixes, all stemming from AI-generated reports. Once again the team is knee-deep in AI bug reports — but this time, they’re not slop. These are bugs that cURL’s regular analysis tools have been completely been overlooking.
“This is new,” says Daniel Stenberg. “It really looks like these new tools are finding problems that none of the old, established tools detect.”
“We regularly run clang-tidy, scan-build, CodeSonar and Coverity on the code, and whenever they find something, we fix it. So when all those tools report zero issues and someone suddenly finds hundreds, that’s pretty spectacular,” he adds — with some understatement.
All the bug reports came from one single developer: Joshua Rogers, an Australian with 15 years in cybersecurity, including at Opera Software in Poland. Today he works in security for a cryptocurrency company.
Over the past few months, he has been evaluating new AI-based tools and has started submitting bug reports to several open-source projects — including cURL, sudo, libwebm, Next.js, avahi, wpa_supplicant and squid.
None of the 50 bugs found in cURL were critical, but Rogers has discovered critical vulnerabilities elsewhere, including in source code from his former employer Opera Software. That bug was patched in early September.
Initially, Rogers hesitated to report bugs to cURL — familiar with Stenberg’s public stance on “AI slop”.
“Even though I could literally see the bugs in the code, I thought there was a 0.001% chance I was wrong — and I’d end up in the hall of shame,” Rogers says with a smile.
But he eventually gathered his courage and started sending reports.
After a while, Stenberg reached out curiously and asked where the reports were coming from.
”After I explained it to him, he asked me to send him the un-reviewed list of problems, and he'd triage them himself.”
“Triage” is a medical term — sorting patients by urgency. In software, it means prioritizing bug reports by severity.
Rogers says he’s received similarly astonished reactions from other open-source maintainers.
On his blog, he has shared insights into how he performs vulnerability analysis using LLM based SAST tools (Static Application Security Testing). His main message: these tools exist, and they’ve become incredibly good.

Comment Re:Recycle everthing possible ffs (Score 1) 109

It maybe a inferior product, but if it takes less energy to generate then starting from scratch you are still coming out ahead.

Many forms of recycling take more energy than working from raw materials. Such recycling is only commercially viable thanks to the high charges for landfill and dumping of used goods.

Comment Re:Recycle everthing possible ffs (Score 1) 109

How is that not recycling?

Because classically, "recycling" means putting it back through industrial processes to manufacture a new item. The extended meaning of recycling to cover reuse, repair and repurposing inadvertently puts carbon-intensive collection and reprocessing of glass on an equal footing with not putting the glass in the bin at all and using it to store things instead of silly plastic Tupperware items.

Comment Re:Perhaps (Score 1) 265

If I do a work for hire whether it is building a deck or writing an app, I get paid and that's it.

If you do a work for hire, there's one person who's paying you. If you are an author, there isn't one person who just pays for everything -- you actually need to sell the book again and again to make back the time you spent on it.

If I create a new product, I get patent protection for 20 years and that's it.

There's a huge difference between copyright and patent.

Patent law was designed to prevent factory machinery designs being kept secret. Factory owners were inventing better and better machines, but keeping them secret so that they would retain competitive advantage. Some designs died with their owners. Patents protection was invented to encourage inventors to document their creations while preventing others from using them, but then to allow the next inventor to create a better machine without being blocked by the patent.

A book isn't a machine or technique, so it's not like technological advancement is hanging on being able to use a copyright work.

(That's not to say I don't think copyright terms aren't too long.)

Comment Re:Won't somebody think of the organizations (Score 1) 265

However, Project Gutenberg does very very little to promote the fact that copyright terms are different in different countries. It would have been pretty trivial to set up the site with an awareness of international terms and dynamically generate correct copyright information for any users from outside of the US.

Slashdot Top Deals

When bad men combine, the good must associate; else they will fall one by one, an unpitied sacrifice in a contemptible struggle. - Edmund Burke

Working...