Comment Not even close... (Score 1) 175
Let's start with some info:
There are 2 sets of rules at the USPS. Those posted on their website and those that exist behind the scenes. There are very specific requirements to complay with their Postal Form 3541 for in county, in area, in state, and out fo state subsections.
Each zip code has to be seperate, except those going out of state, that can be one big bundle (or multiple if there are enough to cross the bundle limit). Each Carrier Route has to be a seperate bundle or bundles. So zipcode 12345 can have several Carrier Routes. One of our local zip codes has 10 different routes. Thats a minimum of 10 bundles. In our case there are 15 bundles for that group. Route C002 is large and in our case has a lot of subscribers, it takkes 3 bundles on weekdays and 5 on Saturday. On Weekdays we can fit 60 papers in a single bundle. We have a max bundle size of 70, in case we have a few extra so we don't need to create a bundles of 2 or 3. Weekends they have recently requested we drop that to 25 per bundle, with limit of 30.
Next there is this called a Firm Bundle. What that is is a bundles of papers for one specific address, like an Apartment complex, or Assisted living facility.
We have a database with over 100000 subscribers/addresses. Many are inactive, many are digital only. Only about 3000 are getting physical papers. That is 3000 addresses we have to account for via the USPS. That is nearly $1500 per day in postage. We can save about $5 per day by using firm bundles. That is what the project was, to take our existing system and start using firm bundles. Per day that isn't much, but after a year that is at least 1 full day of postage eliminiated. So that is what my task was.
My company's owner asked me to use AI to write a project, as a test. I spent a month having the differnet AI's figure out what needs written, then had them build it. Once the base code was written, I had to have the AI fine tune it, over and over and over and over as requested for this test. It was a fairly simple project. Write a label generation system to meet the USPS code for periodicals, including handling bundles, and firm bundles. It had to know what was local, in county, in state, and other (out of state) and properly account for all of it to save us money. One of the last items was generating the proper postal report to report to the USPS. That along took my 2 weeks of 8-hour days dealing with the AI to get ti to work. It kept looking at the exact same set of code over and over. Fixated on it. The problem wasn't there, it was in the data from a previous step but I finally had to tell the AI that it was stupid and to stop making assumptions and check each and every step, one at a time. That didn't work, so I walked the AI through each step, one at a time and it finally found the issue. I told it to look at what it had been doing previously and told it if it can't learn from that then it shoudl format it's own drives and start over.
Total project cost in time, 320 Labor hours by me, nearly 2 BILLIION in AI tokens (used some free, so cost was much less than what it appears), and the code now works but took a lot of human intervention to make it work.
Conslusion:
LLMs are NOT ready to write code without a senior level developer looking over it's shoulder. They get stuck and don't step back to reevaluate unless forced to.
Just a FYI, I had similar issues when trying to find some informartion at Microsoft. I used Copilot (figuring they used their own LLM) to find some information. The information was years out of date. It didn't even check to see if Microsoft's own interface had changed. Filed a complaint with Microsoft over that one, including how it was so behind the times that information it gave me is WRONG and contradicts what is on their website right now.
Az