I was feeding some graphs of my PV production to claude.ai . There was clipping - a flat line near the top - and i was trying to get it to tell me how much more production i would get if the inverter was bigger. It kept insisting there was no flat line or clipping going on, making up the peak number. Confidently. Even after multiple prompts, including some as "are you blind". There are things these tools are just terrible at. Reading graphs is one of them, apparently. I also tried to get it to create a diagram of my PV installation. It did 99% of the job, but just couldn't make very minor formatting corrections - preventing the boxes from overlapping. It tried to regenerate the graph from scratch, but made my unwanted changes in the process.
I have similar issues when using various code generator agents. They struggle enormously with making isolated minor fixes.
It's almost as if we need some non-generative AI to actually get the job done.