Slashdot is powered by your submissions, so send in your scoop

 


Forgot your password?
Close
typodupeerror
×
AI Microsoft

Salesforce CEO Benioff Says Microsoft's Copilot Doesn't Work, Doesn't Offer 'Any Level of Accuracy' And Customers Are 'Left Cleaning Up the Mess' (x.com) 81

Posted by msmash from the not-mincing-words dept.
Salesforce founder and chief executive Marc Benioff has doubled down on his criticism of Microsoft's Copilot, the AI-powered tool that can write Word documents, create PowerPoint presentations, analyze Excel spreadsheets and even reply to emails through Outlook. In a post on X, he writes: When you look at how Copilot has been delivered to customers, it's disappointing. It just doesn't work, and it doesn't deliver any level of accuracy. Gartner says it's spilling data everywhere, and customers are left cleaning up the mess.

To add insult to injury, customers are then told to build their own custom LLMs. I have yet to find anyone who's had a transformational experience with Microsoft Copilot or the pursuit of training and retraining custom LLMs. Copilot is more like Clippy 2.0.
This discussion has been archived. No new comments can be posted.

Salesforce CEO Benioff Says Microsoft's Copilot Doesn't Work, Doesn't Offer 'Any Level of Accuracy' And Customers Are 'Left Clea More

Salesforce CEO Benioff Says Microsoft's Copilot Doesn't Work, Doesn't Offer 'Any Level of Accuracy' And Customers Are 'Left Clea

Comments Filter:

  • Hi there! It looks like you’re frustrated wi (Score:4, Funny)

    by VeryFluffyBunny ( 5037285 ) on Friday October 18, 2024 @08:15AM (#64874279)
    ...Copilot. Would you like help with:

    Throwing your hands up in despair?
    Googling alternatives?
    Creating a meme comparing Copilot to Clippy? (Spoiler: I was way cuter, and I didn’t spill data everywhere!)

    And hey, at least with me, you knew what you were getting—a friendly paperclip, not a half-baked AI with a cleanup crew!

  • Does sales force have its own LLM (Score:3)

    by Revek ( 133289 ) on Friday October 18, 2024 @08:16AM (#64874281)
    I don't follow this bubble very much so I really don't know if they have a competing product. Having said that I will worry about AI when in congeals into something proven.
    • They made a lot of fuss about theirs a few weeks ago during their annual cult/networking/expense-tickets-to-vegas thing. Theirs is called "Einstein" so you know it has to be smart; and they are super all in on paying them to use 'agents' to chatbot customers, along with 'generative' to increase your sales team's spamming efficiency.

      In one sense I can understand why he's rubbishing Microsoft so vigorously: not only does 'copilot' deserve it; MS has a fairly obvious interest and fairly obvious ability(at l

    • Having said that I will worry about AI when in congeals into something proven.

      It works relatively well for doing web searches on Bing. Mostly gets past all the SEO.

    • Re: (Score:2)

      by gweihir ( 88907 )

      Having said that I will worry about AI when in congeals into something proven.

      For LLMs that will never happen. The mathematics used does not allow it. And more broadly, statistical models cannot do reliability. It is not in their nature.

  • You should be using "Einstein AI" [salesforce.com], it's so much better, it says right there this chatbot is so much better than my old one!

    What? No, I don't stand to make billions of dollars from this. How preposterous.

    • The fact that this guy has a competing AI does not automatically make his statement false - it just means you need to look at it with a critical eye.

      • Given that all his statements are his opinions about other people's experiences without any data backing them up, means that those opinions are thus highly suspect and should be assumed to be bullshit.

      • It's Salesforce.

        They are crap all over.

        Their search is also already called Einstein and also already sucks. They are creating suck confusion.

  • First thing to ask copilot (Score:5, Informative)

    by caseih ( 160668 ) on Friday October 18, 2024 @08:26AM (#64874315)

    How do I disable copilot? And it gives a pretty accurate answer to that one. It's literally the only thing I've ever used copilot for.

    • Reminds me of using Edge exactly once (to download Firefox or Chrome). Of course, Edge kicks and screams the whole way in that instance.

      • Re: (Score:2)

        by gweihir ( 88907 )

        Edge is the crappy tool to get you started. Of course "wget" would have been just as useful and nowhere as crappy.

    • Re: (Score:2)

      by gweihir ( 88907 )

      Well, then at least it is useful for one thing. That you would not need without it, but still.

      The two good uses I found for ChatGPT is help with lying ("How to I say xyz in a positive way?) and it actually gives a quite reasonable answer as to why it is not AGI and likely cannot be a basis for AGI.

  • He's not wrong - but there's apparently still an upside to copilot. I've personally not really seen it, but LLMs in general can be helpful for filling out "puff" in documents. Take this comment for example, I could write all of it out, or I could use an LLM:

    You know, folks, Copilot is a tool that can be really, really helpful—tremendous, actually. But let me tell you, it doesn’t always get it right. Sometimes it misunderstands the context and gives suggestions that are just plain wrong. You

    • Re: (Score:2, Insightful)

      by Anonymous Coward

      Sometimes it misunderstands the context and gives suggestions that are just plain wrong.

      Indeed. Some of us have access to Copilot licenses in Teams. I asked it to summarize yesterday's standup meeting (which, coincidentally, nobody remembered to record) and it hallucinated a whole legitimate-sounding summary of key points, actions and blockers... most of which had nothing whatsoever to do with the actual meeting.

    • Re:Not wrong (Score:5, Funny)

      by phantomfive ( 622387 ) on Friday October 18, 2024 @08:51AM (#64874371) Journal

      Take this comment for example, I could write all of it out, or I could use an LLM:

      Just write the prompt into your comment, and save us the time from reading the filler nonsense.

      • Exactly... People use AI to write their resume, companies use AI to pick out the candidates. Just send your prompts! Students use AI to generate their homework, teachers use Ai to grade them. If this AI catches on, all it will do is keep itself busy. Shame of all those nuclear power plants.

      • Yup, "filler" is pointless, don't do it unless the teacher demands it (ie, my high school history teacher who said that I covered all the topics, had all the facts, and had a good argument, but that he wanted 5 more pages). For a comment in code, it needs to be precise, which can mean it has to be long by necessity but not with filler or fluff. That's how you spot the new hires, they stick in fluff.

  • Says the guy that owns a service that attaches HTML files with a rerouter in the head instead of putting the link in the email like a normal person. Apparently he's never heard of Kryptix.

  • CEO = Clueless Executive Officer (Score:3)

    by bradley13 ( 1118935 ) on Friday October 18, 2024 @08:44AM (#64874359) Homepage

    He's a clueless dweeb, who listened to sales pitches from clueless dweebs at Microsoft. He probably hoped he could fire half of his developers. That would really boost his bonus! Turns out that's not the case, so he's disappointed.

    • Many CEOs are clueless, but Marc Benioff probably isn't.
    • It's fun to say it this way be there is nothing but the marketing hype has been huge on AI and has effected corporate stock market valuations to the tune of billions/trllions of dollars. Any CEO not playing along with this takes the risk of being dumped by the board of directors.
  • The best you could hope from microsoft is some half baked mediocer product that they want you to pay for. i generally use chatgpt and never touch google or microsoft biased and weirds llms

  • True conservatism required (Score:4, Insightful)

    by Baron_Yam ( 643147 ) on Friday October 18, 2024 @09:05AM (#64874405)

    Not the political kind... Just "hey, maybe let us not jump blindly into this trend without careful consideration and some reasonable testing".

    "AI" is supposed to be doing all sorts of things that it clearly cannot, and people are losing their jobs to it while we're being told it's magically creating new ones.

    This economic disruption is bad for the average person in the short term, and in the long run it's not great for companies. Of course, it's a lot easier for the companies to change course after a few years, and executives' bonuses won't be affected at all...

  • HIlarious, because he's right (Score:3)

    by Murdoch5 ( 1563847 ) on Friday October 18, 2024 @09:10AM (#64874413) Homepage
    His point is gen AI is a gimmick that doesn't really work, is mashed together patches and a total let down. Can anyone call him out as wrong? Honestly, I don't know because I've never used gen AI that's good enough to warrant an endorsement.

    I have used it to generate scaffolding / boilerplate policy wording, which I then fill in / tailor to my needs, and it does that fairly decently. I've also used it in my IDEs to help with basic boilerplate code generation, and it's maybe 40% accurate at the simple stuff, enough that it saves me time. Would I ever use it in a professional, unguided, unwatched, and unverified capacity? Absolutely not, even accidentally, gen AI is not ready for professional uses cases that a human can't do better or more accurately.

    • Have you tried Cursor?
      It is a very good interface for working with LLMs (with a few added background AI tweaks) that works really well for smaller projects. The backend LLMs are not magically better than before, but the interface integration makes getting/filtering the worthwhile stuff out of them and into your code easy (enjoyable even!).

      Not afflliated with them btw, even though the above sounds like an ad. I use and love Jetbrains IDEs and keep doing so for professional stuff, but have been loving Cursor

      • I have not, I'll take a look. I'm using the AI plugin for WebStorm, and it's good enough. Not only that, but I have ChatGPT for boilerplate generation, and it works well enough that it really does save time writing the scaffolding stuff for policy documents.
  • Now that's how you burn Microsoft's ass.

    • Now that's how you burn Microsoft's ass.

      It's not Clippy 2.0. It's Clippy 3.0. I'm glad everyone forgot Cortana, don't get me wrong...but check out some of the 2016-2019 promo videos from Microsoft about Cortana and all the functionality they were integrating into it, all the way to making a Cortana smart speaker that was going to compete with Alexa. There was an Android app and the whole "tie it to your Microsoft account so Cortana can add plans from e-mails to your calendar"...all that stuff, but like Copilot and Clippy, nobody wanted it.

      The sum

    • Re: (Score:2)

      by PPH ( 736903 )

      Now that's how you burn Microsoft's ass.

      And Bob's your uncle.

  • I can't count the times I've heard people say things in posts and articles that, "I asked an LLM something and it was wrong!" Oh no.

    We're talking to computers here.

    I'll ask you all, 1. How accurate are humans? 2. Do humans accidentally leak data or cause security problems? 3. Do humans understand how to interact effectively with LLMs? 4. Let's check driving safety comparing humans and auto driving AIs. I think you already know the answer.

    1. Humans are not accurate in general and praise their own ac

    • 3. I took a course through a university regarding Prompt Engineering.

      It's kind of hard to take you seriously after that.

      • Really, I find it very interesting that a University would offer this. Presumably to the wider public and not students working toward a degree. It's a difficult concept to wrap your head around without taking the time to learn exactly what it does. And also, emergent behavior from what it does have is already counterintuitive without a lot of mental gymnastics.

        I see very smart people write very bad Google search queries. And some of them even know how modern search engines work. This is at least a few

        • And some of them even know how modern search engines work.

          I don't know how modern search engines work.

          • Yes, I'm dumb. thanks. Great comment. Yeah I haven't worked at MS as a contractor in the 90s, and worked in tech my entire life - on the backend of the net. Yeah, I know nothing about it. Very useful comment, kid.
        • I told the person above that I should have said it was an online course through a university. I'm not a student. I just wanted to understand it better. But, have you noticed the fact that Google syntax is not broken. You cans -"X" and ... they give you a whole lotta X, right? That did not used to be the case. That's all I'm saying. I don't want to dodge ad bullets when I'm looking for a tech solution.
      • It was an online course offered by a university I should have said that. My emphasis wasn't on this.

    • The simple answer is that this is what the marketing says it can do.

      These LLMs do exactly what marketers should be saying they can do. But the moon was promised.

    • Yeah, I would say that you should treat LLMs as a coworker who is possibly incorrect about the stuff he's saying, but has a bunch of experience in the thing you're asking advice about. This mental model leaves you with far more realistic expectations of LLMs.

    • Here's my big problem that your screed reminded me of. The general public quite probably will fall back on the old fault - they trust the computer because computers can't be wrong. You call up to complain about your electric bill in 1990 and are told "the computer can't be wrong". Sure there were a lot of stories like that and most of it was just the same few stories repeated, but there were enough instances of seeing this happen in person that you knew a very large chunk of people were running everythin

  • OK, I'm a dummy. I never got around to disabling automatic updates for the one Win 10 Pro computer I have for work purposes. I recently had the unalloyed pleasure of finding out I had an icon for Co-pilot squatting on my task bar like a turd on the carpet. Fortunately, it seemed easy enough to get rid of. I have a fresh disk image I will revert to if I find out "Uninstall" actually means "Hide and Continue 'Recall' Functionality".

    It's hard to describe the sinking feeling in my stomach when I found that

  • Your first mistake... (Score:3)

    by Hoi Polloi ( 522990 ) on Friday October 18, 2024 @09:24AM (#64874463) Journal

    If you are depending on AI blindly then that is your mistake. These companies shoving AI in your face are as annoying as sites such as FB constantly shoving autocompletion links at you.

  • why does it need to be transformational to add value?

    Nothing of what salesforce has brought to market has been transformational, but some it suits a purpose and thus is adopted.

    For someone to say it's crap because it's not "transformational" is in need of a mirror.

    It's a tool... it's not a replacement for an expert in every field... it's there to assist... if it can make your team slightly more productive, it's a success. And it looks to be a hell of a lot more effective than anything the leadership team a

  • Checking Output (Score:3)

    by John Allsup ( 987 ) <slashdot AT chalisque DOT net> on Friday October 18, 2024 @09:35AM (#64874487) Homepage Journal

    The trouble with AI output is that it needs to be checked by a competent person, and often that checking, if done to a suitable standard, will take longer than doing things the old fashioned way. For coding, AI can be useful in suggesting things to try, but the programmer must understand the output, and to correct any problems. Now if, for example, I'm writing something in Python or Rust, then AI can be a great source of suggestions as to what packages I should look for, and also possible search terms to google/search for. The trouble is that people are lazy, and the checking process will get skimped on.

  • While it's likely that future AI will be a very useful tool, and early versions like AlphaFold are already producing results, today's consumer focused AI offerings are just crap generators that produce stuff that appears to be well written, but is in fact, crap. It's kinda like a BS artist, who confidently claims expertise while spewing nonsense

  • I'm not going to say anything about Copilot itself, but when I go to the article and the criticism is coming from the CEO of a company in the process of launching AI agents that compete with Copilot. In other words he has every reason to be massively biased on this topic and unless he spells out concrete examples rather than vague claims he should not be seen as a credible source.

  • "It looks like you're hyping a bubble."

  • I'm shocked. SHOCKED that M$ created a useless product and shoved it violently upon its entire userbase.

    • It's more about the extended AI hype I think. It's really amazing that there is so much money going into it, and dedicated nuclear power plants for crying out loud.. Yet it sucks and there is really no path for it to have any understanding of anything, which is what it needs. So we are dedicating *nuclear power plants* to spew bullshit. You really cannot make this stuff up.
  • Salesforce has a product line of AI solutions [salesforce.com]. Obviously they're going to talk shit about Copilot because they want people to buy their shitty product instead.

Slashdot Top Deals

One good suit is worth a thousand resumes.

Close