Nah, I mostly write short one off scripts that are literally "just need to work". So far AI has completely failed 5 out of 6 times. I think the issue is not that it can't write an effective script but that the scripts I write are to glue things together in my environment and it has no environmental context.
Our environment has *quirks*. My most recent failure was trying to convert an AD group membership on AD joined Alma computers (using SSSD) into
AI came up with a perfectly good script using getent except for "reasons" SSSD sometimes doesn't actually fill the getent results for a group membership. So in practice in the environment, depending on the computer you ran it on and the phase of the moon, you got null results for group memberships.
Now, stepping back you could argue we should spend time fixing that bug, but all other IT processes just use LDAP anyway. But there's no way for AI to have known that unless I thought to tell it. However, that's also "knowledge" I didn't think to share, and certainly wouldn't expect to put in a extremely long and detailed prompt a la the "vibe coding". If I'm going to psuedo code it out for an hour or two to make sure I provide a lot of context, I might as well have just written the script by then in many cases.
One obvious improvement is that eventually we'll be able to run a powerful model somewhere with sufficient security and data sovereignty and privacy guarantees that we can "ingest all our written docs" and get the AI at least the context a newly hired person would have. IDK when that political issue will happen though so I can't comment on if that "fully fixes" this or not. Most optimistic estimates I've seen from local stakeholders is 18 months.
Ok, but now I've told it to use LDAP. You'd think this would also be well defined and understood, but then AI proceeded to fuck up string mangling and whatever for another hour till I realized I was debugging awk to a level I'd have done if I just was hacking it together myself anyway and called it as any sort of productivity enhancement for me.
Granted, I was fully trying to "vibe code" this because I don't live in AWK and always have to basically iterate test iterate anyway. After I kinda ranted about this in our group chat, a colleague of mine said: "Why don't you just do the filtering and name conversions in LDAP in your initial query?" To which I said something like "whut?". TBH, I personally minimally use LDAP and it's filter format makes my brain melt. But they gave me a 2 line ldapsearch command that did exactly what was needed, that AI had failed to create and in fact had made pages long loops that I know weren't performant... and also didn't end up working at all.
So, the long and short of it is - if getent / SSSD worked 100% of the time, AI would have 110x my script writing. Because it doesn't for "reasons", and fixing that would -100x us as mostly a tangent, in reality it was a wash at best. I'm still going to try to vibe code using the stuff my colleague gave me and wrap the rest of it just to "keep up with AI" but as is obvious this isn't a priority which is why I was vibing around in the first place.
It doesn't bode well though IMO if toy examples yet tied to a real world complex environment have the same issues. I found the one time AI did work for me it was entirely self contained stand alone windows script that didn't interact outside the OSs it was run on.
I won't say that AI won't fix these issues, but IMO fundamentally the limit there is human / business legal / political and how many people want to or can upload all their docs, existing code etc to an AI company, and then can the AIs handle the context size properly.
What they don't have the ability to verify information against an outside source, which is why you can ask it to only give you cases in the Lexis database and still get completely fictitious cases.
I'm pretty sure that's not true given MCP, AI agents, and even just web enabled search. They can check other sources now anyway - but I'm not really sure that would prevent them hallucinating cases without additional training and such, maybe a verifier agent or something. Now of course that it's gotten appealed, they probably did this a couple years ago when AI was far more limited.
https://www.thestudiesshowpod....
Has sources in the list of links on that page. Makes a good case that antibiotic resistance has been overblown over the last 20 years.
I would agree, don't take antibiotics for no reason - but delaying treatment when there's a ticking clock for things like anaplasmosis is probably a worse idea than perhaps taking an unnecessary dose of doxycyclene.
The hardware isn't sold. It is licensed.
All the citations needed.
the AI industry
Just to be a pedantic sod, "AI" isn't just LLMs and things that make images, audio, and the like, it's quite the spectrum actually - including things like medical research.
IMO treating a dynamic and complex industry as a monolith, whatever industry it is, just felt off to me, heh.
Biology is the only science in which multiplication means the same thing as division.