Comment Re:ok (Score 1) 10
Indeed, they take needles in haystacks and make smaller haystacks that have some of the needles. They don't catch some things and they falsely flag a lot of things, but they do either directly or get close to something.
One issue in our codebase that was caught was pretty much spot on. Now upon it being highlighted, it was obvious to anyone that the inexperienced developer screwed up, problem was no one had the attention span to notice. The other issue close to real that it caught was actually not the flaw it indicated, however while looking at the jankiness that caused it to mis-indicate, a real issue was discovered. So it's indicated "fix" would have done nothing and left the codebase vulnerable, but the finding *was* useful in identifying an actual problem and fix.
However, it does open up a gigantic mess in customer engagement. Customers like to use every security tool at their disposal and hold the vendor accountable. Fine, except now we are inundated with false positives because the LLM indicates something, the customer has no way of knowing if the LLM is right or not, and we have to address false positives. We are now having to consider how to "fix" false positives by steering the LLMs away from indicating bogus things when analyzing our product. When it was the occasional misguided security researcher making a misunderstanding, no huge deal, we could discuss nuance and come to an accord. With the LLM mess, there's no skilled human to appeal to, a skeptical customer, and just a lot of volume...