Submission + - New problem: AI finds too many bugs (etn.se) 1
jantangring writes: The open source project cURL used to be flooded with worthless, AI-generated security reports. Over the past few months, those have vanished — replaced by genuinely useful ones. So many, in fact, that the maintainers are struggling to keep up, says Daniel Stenberg, who leads the project.
cURL is not alone.
“I hear similar witness reports from fellow maintainers in many other Open Source projects,” Stenberg writes on LinkedIn.
Several of those colleagues back him up in the discussion thread — among them the maintainers of glibc, Vim, and Node.js.
“Over the last few months, we have stopped getting AI slop security reports in the #curl project. They're gone. Instead we get an ever-increasing amount of really good security reports, almost all done with the help of AI,” says Stenberg.
Stenberg has a straightforward explanation for the shift – better tooling.
“HackerOne did basically nothing new that could explain this (plus, this is mirrored in countless other projects, many of them not on hackerone). This is a notable change in the incoming reports. I'd say it is primarily because the tooling has improved.”
HackerOne is the platform cURL uses to receive bug reports.
There is an unexpected downside to being flooded with good bug reports, though — there are simply too many to handle in time.
“They're submitted in a never-before seen frequency and put us under serious load,” says Stenberg.
The challenge used to be filtering out noise. Now it is keeping pace with reports that actually matter. That is how Steve M. Hernandez, a code security specialist, puts it.
“High quality reports at higher frequency still require the triage capacity and decision consistency to keep up. The bar is moving from filtering noise to keeping pace with real signal.”
cURL is not alone.
“I hear similar witness reports from fellow maintainers in many other Open Source projects,” Stenberg writes on LinkedIn.
Several of those colleagues back him up in the discussion thread — among them the maintainers of glibc, Vim, and Node.js.
“Over the last few months, we have stopped getting AI slop security reports in the #curl project. They're gone. Instead we get an ever-increasing amount of really good security reports, almost all done with the help of AI,” says Stenberg.
Stenberg has a straightforward explanation for the shift – better tooling.
“HackerOne did basically nothing new that could explain this (plus, this is mirrored in countless other projects, many of them not on hackerone). This is a notable change in the incoming reports. I'd say it is primarily because the tooling has improved.”
HackerOne is the platform cURL uses to receive bug reports.
There is an unexpected downside to being flooded with good bug reports, though — there are simply too many to handle in time.
“They're submitted in a never-before seen frequency and put us under serious load,” says Stenberg.
The challenge used to be filtering out noise. Now it is keeping pace with reports that actually matter. That is how Steve M. Hernandez, a code security specialist, puts it.
“High quality reports at higher frequency still require the triage capacity and decision consistency to keep up. The bar is moving from filtering noise to keeping pace with real signal.”
