Comment Re: Subtext is scarier (Score 1) 49
AI is stupid, but it can track details and at least provide a hint.
On the "make stuff that works" side, I've had the experience like yours. If someone lets it go too far, then it's a headache and it's easier to roll it back. If anything a bit painful as it's "empowered" some of the worst to make my day harder, being confidently incorrect all the time.
On the "find stuff that's broken" side, well, humans don't have the attention span and the AI techniques are catching little but critical mistakes by humans. In my codebase that has had regular security reviews for over a decade, including a few outside consulting companies, this year marked the first year that the teams had LLM at their disposal. It found two security issues that no one had noticed, one of which had been there almost essentially from the beginning. Admittedly, neither were exactly world ending (both required attacker to log in with admin privileges and the things you could do were a bit constrained), but they were real and undesirable. One of them kind of missed the point and although it mis-characterized the behavior, it put a light on an area where a human could actually think through and sort out the real issue, and how the flawed approach applied more broadly than the AI identified.
So AI as a review tool can work, it has a significant amount of false positives and misses stuff of course, but it can either directly catch or inspire human attention on a sketchy area.
The security team said that it was actually quite remarkable they only found two vulnerabilities that both required admin access, that most projects they dealt with ended up with 4-5 vulnerabilities exposed to either unprivileged users or even unauthenticated access. This is the sort of code the world is mostly built on.