Salesforce CEO Benioff Says Microsoft's Copilot Doesn't Work, Doesn't Offer 'Any Level of Accuracy' And Customers Are 'Left Cleaning Up the Mess' (x.com) 61
Salesforce founder and chief executive Marc Benioff has doubled down on his criticism of Microsoft's Copilot, the AI-powered tool that can write Word documents, create PowerPoint presentations, analyze Excel spreadsheets and even reply to emails through Outlook. In a post on X, he writes: When you look at how Copilot has been delivered to customers, it's disappointing. It just doesn't work, and it doesn't deliver any level of accuracy. Gartner says it's spilling data everywhere, and customers are left cleaning up the mess.
To add insult to injury, customers are then told to build their own custom LLMs. I have yet to find anyone who's had a transformational experience with Microsoft Copilot or the pursuit of training and retraining custom LLMs. Copilot is more like Clippy 2.0.
To add insult to injury, customers are then told to build their own custom LLMs. I have yet to find anyone who's had a transformational experience with Microsoft Copilot or the pursuit of training and retraining custom LLMs. Copilot is more like Clippy 2.0.
Hi there! It looks like you’re frustrated wi (Score:3, Funny)
Throwing your hands up in despair?
Googling alternatives?
Creating a meme comparing Copilot to Clippy? (Spoiler: I was way cuter, and I didn’t spill data everywhere!)
And hey, at least with me, you knew what you were getting—a friendly paperclip, not a half-baked AI with a cleanup crew!
Re: (Score:3)
Binging* alternatives.
Does sales force have its own LLM (Score:2)
Re: (Score:1)
They either have one or have announced intent to introduce one, I forget which.
If it's as good as Salesforce, it will be a million times shittier than Copilot.
Re:Does sales force have its own LLM (Score:4, Interesting)
If it's as good as Salesforce, it will be a million times shittier than Copilot.
As someone who was forced to clean up one of our git repos at work in which a junior programmer committed dozens of Copilot-generated changes, that's a scary thought.
(And yes, the dude was told he'd be summarily fired if he used that shit ever again without asking permission)
Re: Does sales force have its own LLM (Score:3)
Nobody was ever fired for choosing Microsoft (Copilot).
Re: (Score:2)
Well that's about to change with AI, because you should have seen the mess. We were supposed to ship our new firmware update yesterday and the release date has been pushed back one week just to make sure the cleaned-up code is sane and passes production tests again.
Copilot is not welcome at our company, and we're a Microsoft cloud customer through and through sadly - mostly because our IT guy is utterly incompetent. That should tell you how bad Copilot is.
Re: (Score:2)
...we're a Microsoft cloud customer through and through sadly - mostly because our IT guy is utterly incompetent.
I have found those two things to go together almost exclusively. The deeper a shop is into Microsoft, the more incompetent its leaders tend to be. I witness it personally time after time.
Re: (Score:2)
A junior programmer mis-using a tool doesn't really speak to the quality of the tool.
As an experienced dev (with about two decades in the industry), I use Co-Pilot inside VS Code on a daily basis. At the very least, it's a better autocomplete. Sometimes it's very useful (it basically converted a whole bunch of Yup "DSL" into actual functions instantly), but usually it just saves me a few seconds of typing.
I wouldn't say that it offers a huge productivity increase: if I had to estimate, I'd say it makes me
Re: (Score:3)
No code reviews then, eh? Sorry to break this to you, but your shop sucks.
Re: (Score:3)
In one sense I can understand why he's rubbishing Microsoft so vigorously: not only does 'copilot' deserve it; MS has a fairly obvious interest and fairly obvious ability(at l
Re: (Score:2)
Having said that I will worry about AI when in congeals into something proven.
It works relatively well for doing web searches on Bing. Mostly gets past all the SEO.
Re: Does sales force have its own LLM (Score:2)
This means that Google actually has a way to defeat the SEO crapwave... But it just doesn't want to. Fvck them and their greed.
Re: (Score:2)
Yeah CoPilot is such bullshit (Score:2)
You should be using "Einstein AI" [salesforce.com], it's so much better, it says right there this chatbot is so much better than my old one!
What? No, I don't stand to make billions of dollars from this. How preposterous.
Re: (Score:2)
The fact that this guy has a competing AI does not automatically make his statement false - it just means you need to look at it with a critical eye.
Re: (Score:2)
Given that all his statements are his opinions about other people's experiences without any data backing them up, means that those opinions are thus highly suspect and should be assumed to be bullshit.
First thing to ask copilot (Score:5, Informative)
How do I disable copilot? And it gives a pretty accurate answer to that one. It's literally the only thing I've ever used copilot for.
Re: (Score:2)
Reminds me of using Edge exactly once (to download Firefox or Chrome). Of course, Edge kicks and screams the whole way in that instance.
Not wrong (Score:2)
He's not wrong - but there's apparently still an upside to copilot. I've personally not really seen it, but LLMs in general can be helpful for filling out "puff" in documents. Take this comment for example, I could write all of it out, or I could use an LLM:
You know, folks, Copilot is a tool that can be really, really helpful—tremendous, actually. But let me tell you, it doesn’t always get it right. Sometimes it misunderstands the context and gives suggestions that are just plain wrong. You
Re: (Score:1)
Indeed. Some of us have access to Copilot licenses in Teams. I asked it to summarize yesterday's standup meeting (which, coincidentally, nobody remembered to record) and it hallucinated a whole legitimate-sounding summary of key points, actions and blockers... most of which had nothing whatsoever to do with the actual meeting.
Re:Not wrong (Score:5, Funny)
Take this comment for example, I could write all of it out, or I could use an LLM:
Just write the prompt into your comment, and save us the time from reading the filler nonsense.
Re: Not wrong (Score:2)
orly? (Score:2)
CEO = Clueless Executive Officer (Score:3)
He's a clueless dweeb, who listened to sales pitches from clueless dweebs at Microsoft. He probably hoped he could fire half of his developers. That would really boost his bonus! Turns out that's not the case, so he's disappointed.
Re: (Score:2)
Re: (Score:2)
its microsoft afterall (Score:1, Insightful)
True conservatism required (Score:4, Insightful)
Not the political kind... Just "hey, maybe let us not jump blindly into this trend without careful consideration and some reasonable testing".
"AI" is supposed to be doing all sorts of things that it clearly cannot, and people are losing their jobs to it while we're being told it's magically creating new ones.
This economic disruption is bad for the average person in the short term, and in the long run it's not great for companies. Of course, it's a lot easier for the companies to change course after a few years, and executives' bonuses won't be affected at all...
HIlarious, because he's right (Score:3)
I have used it to generate scaffolding / boilerplate policy wording, which I then fill in / tailor to my needs, and it does that fairly decently. I've also used it in my IDEs to help with basic boilerplate code generation, and it's maybe 40% accurate at the simple stuff, enough that it saves me time. Would I ever use it in a professional, unguided, unwatched, and unverified capacity? Absolutely not, even accidentally, gen AI is not ready for professional uses cases that a human can't do better or more accurately.
Re: (Score:2)
Have you tried Cursor?
It is a very good interface for working with LLMs (with a few added background AI tweaks) that works really well for smaller projects. The backend LLMs are not magically better than before, but the interface integration makes getting/filtering the worthwhile stuff out of them and into your code easy (enjoyable even!).
Not afflliated with them btw, even though the above sounds like an ad. I use and love Jetbrains IDEs and keep doing so for professional stuff, but have been loving Cursor
"Clippy 2.0" (Score:2)
Re: (Score:2)
Now that's how you burn Microsoft's ass.
It's not Clippy 2.0. It's Clippy 3.0. I'm glad everyone forgot Cortana, don't get me wrong...but check out some of the 2016-2019 promo videos from Microsoft about Cortana and all the functionality they were integrating into it, all the way to making a Cortana smart speaker that was going to compete with Alexa. There was an Android app and the whole "tie it to your Microsoft account so Cortana can add plans from e-mails to your calendar"...all that stuff, but like Copilot and Clippy, nobody wanted it.
The sum
Re: (Score:2)
Now that's how you burn Microsoft's ass.
And Bob's your uncle.
The wrong way to use LLMs. (Score:3)
We're talking to computers here.
I'll ask you all, 1. How accurate are humans? 2. Do humans accidentally leak data or cause security problems? 3. Do humans understand how to interact effectively with LLMs? 4. Let's check driving safety comparing humans and auto driving AIs. I think you already know the answer.
1. Humans are not accurate in general and praise their own accuracy over others. Evolution has given us an amazing ability to believe ourselves - regardless of the data that may or may not support our opinions. Even when we are wrong, most have difficult admitting it. Nations have gone to war, and millions have died for what? Tell me about Vietnam, the Japan firebombings, the invention of the atomic bomb and global warming. How are we doing as humans? The other day, I asked a human a question, and it got it wrong.
2. you know the answer. Humans can fairly easily be talked into giving away the keys of the kingdom. We don't at the base really understand how to protect all of our system.
3. I took a course through a university regarding Prompt Engineering. Most people do not understand effective ways to interact with LLMs. So we blame the LLMs for it's failures, when they can be traced to simple ignorance regarding how to interact with LLMs. I know this, because now I work very effectively with LLMs, whereas before I understood effective prompting, and felt like LLMs weren't that great. They are great. People haven't caught up yet. This CEO sounds exactly like someone who expects this wholly new paradigm in computing to be... simple. Yeah. Dunning Kruger anyone?
4. AI driven cars are far, far safer than human drivers. Check the data. But what are people worried about? "Oh no! An AI car had an accident. Anyone hear know the number of people that have been kill by AI driven cars? last I checked it was about... 11. 11. Repeat that to yourself when I tell you that humans die in cars at a rate of 40,000/yr. 40,000, in just the US. If that dichotomy doesn't reach you, you have a thinking problem.
I'm not advocating for AI. I'm advocating that humans regarding AI, learn how to interact with it efficiently like any other tool, before pronouncing failure, as in, "I asked an AI something and it failed." Then what? They stop. That's the trick. Then tell the damn AI that it didn't work and move forward with it. It's responsive and dynamic, unlike some of your friends, and maybe me, or you? Create tight efficient prompts and release them like an agile build, in small prompts one after the other, correcting along the way. Do it step by step.
Here's a situation where I saw this difference. I wanted to learn something about ESP32 boards. I hadn't worked with them. I tried working with the AI and it didn't work out to well. We'd get into "idea loops" and things were stepping over other things and it wasn't working, without going into much detail. But I took the prompt course and realized - I - was doing it wrong. It was my fault. Surprise, surprise. So, then after learning some things, I went back and attempted to do what I wanted with the ESP32 platform again. I succeeded and with very few problems. I used tight and efficient prompts and there wasn't a step that we faced that we couldn't overcome fairly easily. Without the AI, I could not have done it as quickly. As we know, search has become corrupted by predatory advertising making finding things more difficult. Especially with Google having broken their own search syntax - for the sake of predatory ad income.
This CEO sounds like a person who doesn't understand LLMs. Most people don't. They are incredibly useful tools, within their abilities which can be exploited very well, with someone who knows how to communicate effectively with LLMs. So before you get your back up, and point at LLMs and say, "they aren't performing correctly," before you know how to properly use them. If
Re: (Score:2)
3. I took a course through a university regarding Prompt Engineering.
It's kind of hard to take you seriously after that.
Re: (Score:2)
Really, I find it very interesting that a University would offer this. Presumably to the wider public and not students working toward a degree. It's a difficult concept to wrap your head around without taking the time to learn exactly what it does. And also, emergent behavior from what it does have is already counterintuitive without a lot of mental gymnastics.
I see very smart people write very bad Google search queries. And some of them even know how modern search engines work. This is at least a few
Re: (Score:2)
And some of them even know how modern search engines work.
I don't know how modern search engines work.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
The simple answer is that this is what the marketing says it can do.
These LLMs do exactly what marketers should be saying they can do. But the moon was promised.
Re: (Score:2)
Re: (Score:2)
Yeah, I would say that you should treat LLMs as a coworker who is possibly incorrect about the stuff he's saying, but has a bunch of experience in the thing you're asking advice about. This mental model leaves you with far more realistic expectations of LLMs.
A prediction from 1987 (Score:1)
https://youtu.be/VsE0BwQ3l8U?t... [youtu.be]
I caught it from a Win 10 update (Score:2)
OK, I'm a dummy. I never got around to disabling automatic updates for the one Win 10 Pro computer I have for work purposes. I recently had the unalloyed pleasure of finding out I had an icon for Co-pilot squatting on my task bar like a turd on the carpet. Fortunately, it seemed easy enough to get rid of. I have a fresh disk image I will revert to if I find out "Uninstall" actually means "Hide and Continue 'Recall' Functionality".
It's hard to describe the sinking feeling in my stomach when I found that
Your first mistake... (Score:2)
If you are depending on AI blindly then that is your mistake. These companies shoving AI in your face are as annoying as sites such as FB constantly shoving autocompletion links at you.
why does it need to be transformational to add.. (Score:2)
why does it need to be transformational to add value?
Nothing of what salesforce has brought to market has been transformational, but some it suits a purpose and thus is adopted.
For someone to say it's crap because it's not "transformational" is in need of a mirror.
It's a tool... it's not a replacement for an expert in every field... it's there to assist... if it can make your team slightly more productive, it's a success. And it looks to be a hell of a lot more effective than anything the leadership team a
Checking Output (Score:3)
The trouble with AI output is that it needs to be checked by a competent person, and often that checking, if done to a suitable standard, will take longer than doing things the old fashioned way. For coding, AI can be useful in suggesting things to try, but the programmer must understand the output, and to correct any problems. Now if, for example, I'm writing something in Python or Rust, then AI can be a great source of suggestions as to what packages I should look for, and also possible search terms to google/search for. The trouble is that people are lazy, and the checking process will get skimped on.
As expected (Score:2)
While it's likely that future AI will be a very useful tool, and early versions like AlphaFold are already producing results, today's consumer focused AI offerings are just crap generators that produce stuff that appears to be well written, but is in fact, crap. It's kinda like a BS artist, who confidently claims expertise while spewing nonsense
Salesforce AI agents (Score:1)
Clippy (Score:2)
"It looks like you're hyping a bubble."
Well spank my ass and call me Charlie (Score:2)
I'm shocked. SHOCKED that M$ created a useless product and shoved it violently upon its entire userbase.