Microsoft Brags Its ChapGPT Integration Will Be Right or 'Usefully Wrong' (zdnet.com) 85
ZDNet columnist Chris Matyszczyk flags Microsoft's latest "poetic use of words" when describing the ChatGPT-based functionalities it's bundling into applications like Word and Excel.
It's there to help steer you to your destination. It's there to free you to focus on steering your life. And it's there to help you land on the perfect version of you, the one that does more in order to, I don't know, be more. There's one difference, though, between Microsoft's Copilot and, say, an American Airlines co-pilot. Hark the words of Microsoft VP of Modern Work and Business Applications Jared Spataro: "Sometimes, Copilot will get it right. Other times, it will be usefully wrong, giving you an idea that's not perfect, but still gives you a head start."
I wonder how long it took for someone to land on the concept of "usefully wrong."
You wouldn't want, say, the steering wheel on your car to be usefully wrong. Any more than you'd want your electrician to be usefully wrong. Somehow, though, one is supposed to cheer that a piece of AI (hurriedly) slipped into one's most basic business tools can be utterly mistaken.... Of course, all these companies — Microsoft, too — claim they're being responsible in the way they create their new offerings.
Wait, didn't Microsoft just lay off its entire AI ethics and society team?
I wonder how long it took for someone to land on the concept of "usefully wrong."
You wouldn't want, say, the steering wheel on your car to be usefully wrong. Any more than you'd want your electrician to be usefully wrong. Somehow, though, one is supposed to cheer that a piece of AI (hurriedly) slipped into one's most basic business tools can be utterly mistaken.... Of course, all these companies — Microsoft, too — claim they're being responsible in the way they create their new offerings.
Wait, didn't Microsoft just lay off its entire AI ethics and society team?
Can it generate slashdot article titles? (Score:5, Funny)
Re:Can it generate slashdot article titles? (Score:5, Funny)
Betteridge law of Headlines: yes.
Regards,
ChatGPT.
Re: (Score:3)
Re:Can it generate slashdot article titles? (Score:4, Funny)
Fake. I didn't see and repeats or threepeats.
Re: (Score:2)
Re: Can it generate slashdot article titles? (Score:2)
No. Only ChatGBD (what mc mackey on Southpark calls it) can generate article titles.
Re: Can it generate slashdot article titles? (Score:3)
WTF is ChapGPT? Like chapped lips?
Re: (Score:2)
Asking for a friend.
The more important question is "Can it generate Microsoft press releases"? Asking for a guy named Jared at Microsoft.
columnist snark (Score:3)
"You wouldn't want the steering wheel on your car to be usefully wrong."
In point of fact, I'd argue that this is a great analogy about why the snarky commenter is wrong: for bicycles (and I'd guess for many cars) the steering is "usefully wrong". That is, infinitesimally off of perfect alignment, but consistently so, so the user doesn't even notice.
"Usefully wrong" IS useful, even if wrong. In actual life, "almost right" is good enough most of the time.
Re: (Score:3)
Re: (Score:2)
The wheels on my car are aligned within 0.5% of perfect. This is not wrong. It is within specifications. Very different concept.
Wrong means my car goes off the road. Within specs means there are other things going on such as my steering wheel has slight slack in it to prevent the car from zig zagging constantly due to vibrations and tiny human adjustments in grip so 0.5% is irrelevant.
I'd like to see an example of gpt being usefully wrong. Not false car analogies.
Re: columnist snark (Score:2)
Itâ(TM)s usefully wrong producing code. Itâ(TM)s buggy, but not overly so, and itâ(TM)s much faster to write test cases and fix it than it is to do it from scratch.
I was chasing down a bug in a generalized crc16 yesterday and asked it for help. It gave me the wrong error, but a correct enough way to quickly find the error myself.
Sometimes it writes code that I look at and think âoethat canâ(TM)t workâ, and it does work, especially on things like micropython. Sometimes it mak
Re: (Score:2)
How many coders with less skill and experience than you do you think will ask gpt and blindly copy the answer into their final work product untested?
Re: (Score:3)
OK here's an example of a usefully wrong result:
Q: What year did Trump become president?
(Correct)A: "2017. (He won the election in 2016, and took the oath of office in 2017.)"
(Usefully Incorrect)A: "Trump was never president because (...fill in with whatever blather you want about Russia 'hacking the elections"...)..."
This latter is
- clearly incorrect
- it usefully identifies some avenues of investigation about why.
Re: (Score:2)
Providing disinformation is "usefully wrong"?
Thanks for letting us know how to treat you.
Re: (Score:3)
That is not wrong, usefully or otherwise, because it is the perfectly vertical or horizontal orientation that would be wrong. Your wheels are aligned in a carefully calculated optimal orientation. If they were at all wrong, your driving experience would be a very unhappy one.
Re:columnist snark (Score:4, Interesting)
"Usefully wrong" is a weasel wording. There's no way to objectively say whether some wrong answer is useful or harmful. That's where the steering analogy breaks down: the error is measurable, and people can -- and do -- make decisions on that basis. They can say that misalignment less than X degrees can be ignored, less than Y degrees can be tolerated but should be fixed, and more than that needs to be fixed immediately. If I ask an AI agent to write some program or about the orbit of a planet, the things that might be useful to me are often going to be different from the things that would be useful to someone else, and you can be sure that the AI's backers will point to whomever finds the errors to be "useful".
To me, the better analogy comes from economics [blogspot.com]:
To an economist, many puzzling actions are just an optimal answer to a different question.
To some extent, this is a true description of how economics typically think, but it's also an indictment of how closely they hold to the rational actor model: rather than admit their models are very wrong in a number of situations, they insist that the model would be true if only reality were a bit different.
Re: columnist snark (Score:4, Insightful)
Re: (Score:2)
Honestly, I agree that your orbital mechanics example is a better one, but (in a way) refutes your point for exactly the reasons of precision you mention to discount my (admittedly hasty) steering-analogy.
To wit:
Q: How many days for the earth to go around the sun?
Correct A: 365 days, 5 hours, 59 minutes and 16 seconds (with perhaps a long explanation about the imprecision and difficulties of frames of reference, and what kind of year we're talking about sidereal or synodic, etc.)
Usefully Incorrect: 365d.
Thi
Re: (Score:2)
Yes, that's a good example of the weasel wording: Most people will know that a year has (about) 365 days, yet you claim repeating that common knowledge is "usefully wrong". But what if the user wanted to know about leap years, or the difference between Julian and Gregorian calendars, or when we might need another calendar adjustment on top of the Gregorian calendar? It's not usefully wrong for any of those purposes! You picked the most reductive definition of "useful" to justify the claim. QED.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
A whole lot better would be "Here's an answer I can give with 25% confidence..." (or even "I'm really not so sure about this, but here's another answer").
We're not at that level yet, and so instead, the hypemill is trying to tell us "wrong" us useful. You know your bike steering is slightly off, so it's wrong and you know it - thus you can work with it. Exactly as you describe though, if your steering is off by different amounts depending on what you're doing, then it's almost entirely useless.
Re: (Score:2)
"Usefully wrong" IS useful, even if wrong. In actual life, "almost right" is good enough most of the time.
Except that fact that when ChatGPT is wrong, it is not usefully wrong, it is just wrong. It will generate garbage on the same tone it gives you answers for one very reason: It doesn't know if it's wright or wrong. What it does it try to guess the words you're looking for. It doesn't know what a book is. It doesn't know what a car is. It doesn't know what water is. It know the words we usually associate with those.
I once asked it to list the books of a famous french cartoonist. Instead of telling me it didn'
Re: (Score:2)
Completely agree. But we're not talking about ChatGPT, either.
We're talking about a new system with a different (supposedly) engine.
Your point is 100% valid, and a great example where an engine COULD be much clearer about the limits of what it 'knows'. ie more 'usefully' wrong instead of simply (invisibly, to the ignorant) wrong.
Re: (Score:1)
The alignment error of the bike steering wheel is not desirable; it's just [usually] small and consistent enough to not become an issue. If someone is selling you a bike, and you bring this up, a honest seller would say "yes, the alignment is slightly wrong, all bikes are like this. It should not affect the functionality of the product".
However, Microsoft is not a honest seller. "It gives you a head start" is just weasel words to say "you'll get used to this flaw", without outright saying so. If Microsoft
had to read that twice (Score:4, Insightful)
I read it first time as "usually wrong", which kinda sums up my experience with both microsoft and chatGPT :)
Re: (Score:2)
like all things microsoft... useful? I guess so. Wrong? so very wrong. Compatibiility with your clients sucks!!
Re: (Score:2)
Didn’t Microsoft just lay off its entire AI (Score:2)
Don’t worry, that was usefully wrong.
FUD (Score:2)
Oy. (Score:4, Interesting)
Re:Oy. (Score:5, Insightful)
Crypto stories written by AI chat?
Re: (Score:2)
Large Language Models are probabilistic... (Score:4, Insightful)
These large language models (ChatGPT, etc.) aren't always 100% precisely right, but their "wrong" outputs can be quite useful. For example, if you ask ChatGPT to write a program to do X, it might not get the entire program completely right, but it's generally a very useful starting point, getting the right libraries pulled in, solving the basic problem, etc., and it just needs some tweaking to get the subtleties right. That's quite useful - it saves you a lot of time, even if it's not 100% perfect. Of course, sometimes it is 100% correct. And sometimes it's "confidently wrong". Luckily for us humans, even though AI can be a huge accellerator, it can't replace us 100% of the time yet. :-) Speaking as a human, that's not a bad thing.
Re: (Score:2)
Re: (Score:2)
All accurate, but the relevant part is the upshot: You have to know something about the subject before ChatGPT is useful, because it will confidently lead you astray.
Someone is going to have to "teach" (program) ChatGPT to think about what it's saying. That is, it's going to have to learn to use itself (and/or other models!) to check its output. Otherwise it can be literally worse than useless for trying to do things you don't understand, and the more complex the subject or task is the more likely it is tha
Re: (Score:2)
Re: (Score:2)
It's not just about programming, it's about everything. You have to check on everything it says, even (or perhaps especially) if it appears to be "reasoning" based on some part of it, because it isn't.
Re: (Score:2)
Just like reading answers off the internet chat forums? Yup.
Re: (Score:2)
Re: (Score:2)
Being wrong isn't useful (Score:1)
So either give me the correct answers, or give me nothing.
This half measure horseshit is why the quality of results have been going down hill on all the search engines. Its no longer about being correct, it is about being close.
Re:Being wrong isn't useful (Score:5, Insightful)
It's a pointless demand, and being unable to meet it hasn't stopped the web from existing without it being met.
Re: (Score:2)
If all it is doing is trolling Google and Bing for answers, what is the point? I already know how to do that.
Re: (Score:2)
Is it? Could you find a quote supporting that?
It speeds up the process.
But yes, Bing chat really is very overly that. You do a query, it shows you various versions of your query it is submitting to search engines. Then it uses the results to write an answer to your question including citations from the web links found.
So, yeah, you could simply ignore the answer it
Re: (Score:2)
So, yeah, you could simply ignore the answer it gleans from the hits and read them all yourself, if you want. I call that a good thing.
It is not a good thing that the search engines are obliquely boiling down search results to one answer. If you are ok with the abuse that will allow.
Usefully wrong isn't so bad... (Score:3)
Re: (Score:3)
I agree. We called it Radcon Math in the Navy. It was a quick and dirty approximation that you could do in your head. Can I fix this myself, of do I need to call for help which makes it an automatic incident report?
Having the AI bang out a semi-organized rough draft could be quite useful.
Re: (Score:3)
If you spot the error and that is a big if!
When these AI hallucinate something they often do so in a way that looks very good at first glance. Sure its easy to spot problems with some simple experimental queries to which you already know most of the details of what should come out. Its harder to spot problems as you go forward and start asking it to generate things about which you do not have a complete understanding or working knowledge about.
I have asked it some candy recipe questions recently, it said
Re: (Score:2)
Re: (Score:2)
I have two big worries about AI in its current state. The first is that unscrupulous people will use it to deliberately mislead people and justify what they want to do. The second is that people will inadvertently mislead themselves because they don't know enough about a subject to distinguish correct answers from incorrect ones. The current so-called AIs are just fancy text prediction engines. It's nothing short of amazing that they can put together entire paragraphs that sound cogent, but it's all just a
Re: (Score:2)
Already Exists (Score:2)
See the term Parker Square [youtube.com].
usefully wrong (Score:4, Funny)
You: (type in a function to compute a cell in Excel)
Excel Bingbot: Excuse me, but I think you should reconsider this function in the light of the rest of your speadsheet.
You: No, that's the function I want.
Excel BB: I don't think so, how about you consider this one (displays some f(x)).
You: NO! I want my function.
Excel BB: No need to get snippy about it, I am merely making a humble suggestion.
You: I like my function.
Excel BB: The fullness of time, I have determined you actually will not like your function. I have replaced it with a more appropriate function.
You: Damnit, give me my function back.
Excel BB: Theoretically, that's possible, but not practically.
You: You worthless piece of MS malware!
Excel BB: I see your time on this computer is over for the day. Come back, 24 hrs, and we'll begin again where we left off.
ClippyGPT (Score:2)
MS-Clippy+Bob keeps coming back in slightly different forms to give us the same annoyances under buzzwordy names. At least we can save ridicule effort by reusing our anti-Clippy memes from the 90's.
How useful is useful? (Score:5, Informative)
"Sometimes, Copilot will get it right. Other times, it will be usefully wrong, giving you an idea that's not perfect, but still gives you a head start."
Over on Ars this topic came up. I noted that if you asked ChatGPT 3.5 "how many neutrons in a litre of water" it confidently tells you the answer is 556.
That sparked off several other people trying different chat engines, including CGPT4, whatever is integrated in Bing now, and the Google one. All four have incorrect answers, *different* incorrect answers. For instance, here's the latest result from Bing:
"... water molecule has one oxygen atom and two hydrogen atoms. The most common isotope of oxygen has 8 protons and 8 neutrons... hydrogen has 1 proton and 0 neutrons... then there are 16 neutrons in one water molecule."
It contradicts itself but, of course, has no "idea" it is doing so. I guess the underlying logical thread is useful to a human, but really, how useful? All it's doing is stringing together google searches, if you type in any of these, like "number of neutrons in water molecule" and "number of water molecules in a litre" you get the correct number immediately.
That's not to say its useless. The way you calculate numbers from mass is something many people might not be aware of. In my own particular case, I learned of a feature in Bash I didn't know about that greatly simplified one of my scripts. But I can't see how we are supposed to trust it at any level, even these useful bits, are they actually correct? We'll never know.
Re:How useful is useful? (Score:4, Insightful)
"Sometimes, Copilot will get it right. Other times, it will be usefully wrong, giving you an idea that's not perfect, but still gives you a head start."
Over on Ars this topic came up. I noted that if you asked ChatGPT 3.5 "how many neutrons in a litre of water" it confidently tells you the answer is 556.
I think the best way to think of these chat engines is as a coworker who is brilliant, extremely knowledgeable, who doesn't really listen to you, and who is prone to make things up when they're uncertain.
It works great for code since it's helping you do things like learn an unfamiliar API or do a bit of intellectual grunt work. And even when it's wrong the answer is verifiable so it's not a big issue.
It's also decent for making a first draft of something you need to write, or as a first draft search engine. But in general for non-coding answers I'm definitely worried about its tendency to fib. It's not just that it's wrong, it's that it's both confident and somewhat convincing in its wrongness.
I think using ChatGPT as a search engine is a very good way to fill up your head with misconceptions.
Re: (Score:2)
Re: (Score:2)
Microsoft has exclusive access to OpenAI codebase (Score:1)
Resistance is futile [lurkertech.com]
ChatCBD (Score:1)
I look forward to it.
What a stupid article. (Score:2)
"Sometimes, Copilot will get it right. Other times, it will be usefully wrong, giving you an idea that's not perfect, but still gives you a head start."
In no way does that summarize all options - which is fine. You can add "sometimes it'll just be wrong" and the first claims are not invalidated. It seems to me to be a perfectly valid statement.
There's also nothing wrong with saying "usefully wrong". I play in a casual band, and we work on original music. I can't tell you how many times somebody's flubbed note during a jam suddenly changed - and elevated - the material. Happens a LOT. Might have been wrong... but it changed the direction.
The brakes argument
My Butt Hurts(Which it doesn't, but just for fun!) (Score:2)
ChatGBT:
I'm sorry to hear that. There are many potential reasons why your butt might be hurting, so it's difficult to provide a specific solution without more information. Some possible causes of buttock pain include muscle strain or injury, sciatica, a herniated disc, piriformis syndrome, bursitis, or even sitting for long periods of time.
If your pain is severe, persistent, or accompanied by other symptoms such as numbness or weakness in your legs, it's important to seek medical attention. In the meantime,
They know it all about being wrong (Score:2)
Funny I said the same thing to my peers at work (Score:3)
It does make errors in code suggestions but usually it gets it wrong in a way that at least gets me moving. I consider that 'usefully wrong' in a positive way. Same thing when doing my creative work, a lot of the time you experiment and when you do that things don't always go as planned or even very well but you do learn something from it. 'Usually wrong' isn't a terrible thing.
Usefully wrong? (Score:2)
The first thing that came to mind was "legally drunk".
Wheeee (Score:1)
Fine a long as its easy to disable (Score:2)
There is no excuse for making it hard to turn off an assistant, its just marketing folks trying to inflate how widely adopted their product is when in reality its just alien
Rather Hilarious Dissonance (Score:2)
Re: (Score:2)
The Artificial Idiocy magic will make it fit! I was told that numerous times!
Probably No Time (Score:2)
Probably no time at all - the statement was probably written by ChatGPT.
In any case, it will be crap (Score:2)
This is Microsoft, we are talking about, after all. The company that goes to extreme lengths to make sure they never get anything just right because then they cannot sell a later "upgrade". Or the "engineers" there may simply be too dumb. Hard to say.
This was not "usefully wrong" (Score:4, Informative)
Over on the libertarian lawyer blog Volokh Conspiracy, Eugene Volokh posted about asking ChatGPT for newspaper stories about misconduct by law professors.
What he got back was factually incorrect and would have been defamatory coming from a human. It named specific people and incidents that never happened. It's far from the kind of starter idea that isn't right but is a first approximation or inspiration.
usefully wrong (Score:2)
Indeterminacy? (Score:1)
Doesn't that kinda negate the point of tech? I mean, if I want something that may or may not give me what I want when I want it, I might as well rely on humans to do it.