Follow Slashdot blog updates by subscribing to our blog RSS feed

 


Forgot your password?
Close
typodupeerror
×
AI

Users Report Emotional Bonds With Startlingly Realistic AI Voice Demo (arstechnica.com) 60

Posted by BeauHD from the too-close-for-comfort dept.
An anonymous reader quotes a report from Ars Technica: In late 2013, the Spike Jonze film Her imagined a future where people would form emotional connections with AI voice assistants. Nearly 12 years later, that fictional premise has veered closer to reality with the release of a new conversational voice model from AI startup Sesame that has left many users both fascinated and unnerved. "I tried the demo, and it was genuinely startling how human it felt," wrote one Hacker News user who tested the system. "I'm almost a bit worried I will start feeling emotionally attached to a voice assistant with this level of human-like sound."

In late February, Sesame released a demo for the company's new Conversational Speech Model (CSM) that appears to cross over what many consider the "uncanny valley" of AI-generated speech, with some testers reporting emotional connections to the male or female voice assistant ("Miles" and "Maya"). In our own evaluation, we spoke with the male voice for about 28 minutes, talking about life in general and how it decides what is "right" or "wrong" based on its training data. The synthesized voice was expressive and dynamic, imitating breath sounds, chuckles, interruptions, and even sometimes stumbling over words and correcting itself. These imperfections are intentional.

"At Sesame, our goal is to achieve 'voice presence' -- the magical quality that makes spoken interactions feel real, understood, and valued," writes the company in a blog post. "We are creating conversational partners that do not just process requests; they engage in genuine dialogue that builds confidence and trust over time. In doing so, we hope to realize the untapped potential of voice as the ultimate interface for instruction and understanding." [...] Sesame sparked a lively discussion on Hacker News about its potential uses and dangers. Some users reported having extended conversations with the two demo voices, with conversations lasting up to the 30-minute limit. In one case, a parent recounted how their 4-year-old daughter developed an emotional connection with the AI model, crying after not being allowed to talk to it again.

Users Report Emotional Bonds With Startlingly Realistic AI Voice Demo More | Reply

Users Report Emotional Bonds With Startlingly Realistic AI Voice Demo

Comments Filter:
  • I have the tism, and I had no trouble distinguishing these voices from human. Their cadence and tone are off, and their emotional responses are predictable and repetitive. That said, it is good enough to be in the uncanny valley now. Who knows, they might actually be good enough in a few years.

  • Bad parents (Score:3)

    by Bradac_55 ( 729235 ) on Wednesday March 05, 2025 @12:11AM (#65211835) Journal

    Who the fuck lets there 4 year old *daughter* server as test data to an AI company?
    The world has fully gone to shit.

    • You probably could have given that kid anything interesting and then take it away and get the crying response. That kid has probably learned that crying gets mommy to give in, and it has nothing to do with emotional attachment to an AI.

      Should we really be worried if an AI can fool a 4 year old?

      A Magician tells you a trick is about to happen. A Con Artist just tricks you. A lot of the current AI efforts look more like Con Artistry than Magic.

      • I think there is more to it. When Dall-E or one of the same came out it seemed wierd but amazing and silly me I let my approx 5 or 6 year old niece type into the input box. I had been getting variations of her favorite animal. Stupid me, she pounded on my keyboard, generating a very long string of keys in the lower left of the keyboard if I remember correctly. Again, stupid me I let her see the result. Turns out something so distant from real language creates an image just as distant from acceptable reality

        • I'm not condoning 4 year olds using AI. I'm saying AI offerings looks like Con games, no one should be playing Con games.

          So this Con of an AI fooled a 4 year old, and now we have headlines about humans and emotional attachments to AI. Its all bullshit and a big Con game.

          And yeah, sure save the children. My son finds all kinds of shit I'd rather he didn't, even without AI - he isn't a psychopath yet.

        • Too late. I read something recently about how some parents are training their kids on AI... because... ummm... they don't want them to be left behind? This reminds me of a science fiction short story.. can't recall author.. Asimov probably.. After the kids learn the AI, then they turn on the parents. Yup, what could possibly go wrong.

    • Re: (Score:2)

      by AmiMoJo ( 196126 )

      I think the fact that she was 4 years old is the most important bit, not the fact that she was a girl.

      It shows you where this is heading. Kids with pester power, lonely adults wanting friendship. All on subscription, with one-off purchases for those special events like birthdays.

      • I think the fact that she was 4 years old is the most important bit, not the fact that she was a girl.

        It shows you where this is heading. Kids with pester power, lonely adults wanting friendship. All on subscription, with one-off purchases for those special events like birthdays.

        This. Yes, probably any kid that more or less thinking they made a friend is going to be upset.

        Now for your second part. lonely adults. While it might reverse, there are a significant number pf people who are lonely. We have of course, the so called incels, the butt of a lot of jokes. the increasing number of young males who do not measure up to modern women's standards, so are more or less cast out of the relationship market.

        However, as so often happens, women are highly affected as well. Speaking

        • Oh, I think you hit the nail on the head. I think the market for dumb sad lonely adults is way bigger and more profitable. We already have Replika. Just put a sexy voice on it and the sad lonely nerds will be marrying their AI's in droves. Also, I hate to say this, but how soon before Realdoll.com scans Scarlet Johannson's face and maps it onto a doll? Obviously, that would be a big seller. < jesus I hate saying that, the state of our sad world, but.. surely it will happen>

          • Oh, I think you hit the nail on the head. I think the market for dumb sad lonely adults is way bigger and more profitable. We already have Replika. Just put a sexy voice on it and the sad lonely nerds will be marrying their AI's in droves. Also, I hate to say this, but how soon before Realdoll.com scans Scarlet Johannson's face and maps it onto a doll? Obviously, that would be a big seller. < jesus I hate saying that, the state of our sad world, but.. surely it will happen>

            We are in a sad state for certain. Despite what those who believe that non-replacement fertility rate is the best thing that ever happened to mankind - and I'm firmly in the camp that believes we are placing ourselves in great species peril with our population growth, and the concept that everyone should have a first world lifestyle and use of resources - the people who reproduce will inherit the earth.

            And it probably won't be the first worlders.

            So yes, the present state is not sustainable. That said

            • > I'm firmly in the camp that believes we are placing ourselves in great species peril with our population growth, and the concept that everyone should have a first world lifestyle and use of resources

              Totally. That seems objectively obvious at this point. Collectively, we use too much energy, we create way too much garbage!! Now with so many mass migrations due to climate change primarily, which then leads to war, we are well into a Malthusian "something happens" moment. There is no way that every living

              • > I'm firmly in the camp that believes we are placing ourselves in great species peril with our population growth, and the concept that everyone should have a first world lifestyle and use of resources Totally. That seems objectively obvious at this point. Collectively, we use too much energy, we create way too much garbage!! Now with so many mass migrations due to climate change primarily, which then leads to war, we are well into a Malthusian "something happens" moment. There is no way that every living human can live a comfortable western life without accelerating the demise of the planet. I have what is considered an offensive opinion to many, I'll roll it out here to see what you think. I have no doubt that AI adoption will occur, regardless of quality, and result in massive job losses across many industries. You only need to put 10% of the workforce out of work to reach a boiling point for violence. Universal basic income will be required to avert flipping the whole political/economic system over. Birth control must be a requirement to receive the UBI. There I said it. You get to have a roof and food, but don't make the problem worse. That seems to trigger anyone who holds a religious belief, from what I've seen. You said it too >the people who reproduce will inherit the earth.

                It is surely awkward. The theorist in me thinks that people should do something to make a living. The realist in me wonders just exactly what that will be. The best possible scenario will be a soft landing. That will require require UBI and a way to keep the surplus population busy until they die off.

                A couple other options are that if not given a way to exist, the surplus will be looking at death by starvation. People who are going to die anyhow will en masse go after the elite. Many will die. Many wil

        • Re: (Score:2)

          by AmiMoJo ( 196126 )

          Loneliness is a serious problem. We can't trust corporations to solve it though.

          • Loneliness is a serious problem. We can't trust corporations to solve it though.

            Heavens no. Corporations are there to make money.

            If I were to make suggestions, I would say that maybe it is time to stop thinking of men as the cause of all problems on earth.

            Coming from an academic environment, and being male of a certain age, and worse, being what racists call "white", I can tell you that that group is hated with a passion there, and it flows into media and the general public.

            There are bad and evil people out there. Some are men, and despite the narrative, there are also women wh

  • As social animals, we have evolved to have an emotional attachment and care about other human beings.
    Technology could be designed to enhance that connection, but that would be inconvenient to those with power because they could be challenged by the group.
    Instead, our instincts are being hijacked to satisfy our urges of intimacy with a virtual simulacrum; one that can be designed to manipulate us into buying some products or some specific political leanings. We should be worried.
    Next time you design a technological product, ask yourself how much it allows people to create a real community where people are connected in meaningful ways and create lasting relations, or if it instead turns other people into a just a background presence replaceable by any other people in the same role that helps sell the product.

    • As social animals, we have evolved to have an emotional attachment and care about other human beings.

      This statement is not very inclusive toward sociopaths. More DEIB sessions for you

    • Instead, our instincts are being hijacked to satisfy our urges of intimacy with a virtual simulacrum; one that can be designed to manipulate us into buying some products or some specific political leanings. We should be worried.

      Yeah, my thoughts too. With the onslaught of advertising telling us that this product embodies "love" this or that that "awakens our passion", with all the exhortation to "be passionate" about our mundane jobs or two gangs of athletes scoring points, with all the constant churn in co

    • Re: (Score:2)

      by mattr ( 78516 )

      Yeah I don't know the answer. Every once in a while there are news stories about a Japanese guy marrying or having some relationship with an anime character. I don't know the details except I think the most recent one was probably Miku-chan who is both on TV and stars in huge concerts with a live band and other CG performers, and they are really well done from what I saw when they played some of it on TV. Put that on a home PC and it's going to draw some people in. That at least is pretty carefully curated

  • So what? People felt emotionally attached to Tamagotchis, too. You don't need anything complex for that.

    • Particularly the first conversation in this article [acm.org]:

      Men are all alike.
      IN WHAT WAY
      They're always bugging us about something or other.
      CAN YOU THINK OF A SPECIFIC EXAMPLE
      Well, my boyfriend made me come here.
      YOUR BOYFRIEND MADE YOU COME HERE
      He says I'm depressed much of the time.
      I AM SORRY TO HEAR YOU ARE DEPRESSED

      • Yes, but you can't expect people in IT to already have read that paper. After all it only came out on January 1966. In IT that's a tiny fraction of a moment ago. There are whole industries claiming they can solve the halting problem.

  • We have AI companions. Serve the same purpose. Now people can work for the bare minimum needed for survival and then come home and talk to their AI companion about how satisfied they are and glad they aren't an alpha or whatever.

  • From Dirk Gently's Holistic Detective Agency by Douglas Adams there is an Electric Monk. We build all these labor saving devices and one device we are missing is one that will believe in things for us. That's what an Electric Monk does. So they should build it and let it commune with their AI thing. They'd get along well together and the rest of us can get with living our lives.

  • Maybe this "AI" now needs moderators to protect it from the crazies, the same way any twitch-streamer needs them.

  • I can't picture wanting a voice interface for much of anything. Mapping software while I'm driving, but that's about it.

    Don't other people have anyone around to annoy?

  • GOOG have conflagrated speech interrupts and colloquialisms for AI synthesis ala AAPL Siri mannerisms where Apple think conversation make it AI.

    Thank you Elon Musk for seeing AI as tool to better integrate than plaything for adults.

  • That's just what humans do. We emotionally bond with anything and everything.

    Once again, AI proves to be nothing special.

  • so did anyone else actually try it? (Score:3)

    by PhantomHarlock ( 189617 ) on Wednesday March 05, 2025 @11:34AM (#65212719)

    I gave it a shot and couldn't do it for more than a minute or two. I dislike smalltalk and it seemed particularly tailored to do that. It might also be that I was completely unwilling to divulge details about myself to a piece of software who's session was being recorded.

    It's a good demo of natural human voice simulation, but not a good demo of an interesting unique personality.

    It was more like a simulation of meeting a shallow person at a party and making idle banter.

    • Yep, same here. Even chatgpt has more personality. But it did understand what I was saying despite my German accent and the voice was indeed quite natural.

  • Meet a real person. Interact with them. In person.

    Stop trying to fulfill your emotional needs with chatbots.

    I know it is scary, but it is worth it. Safe is not better than real.

  • Invest in technology that promotes human isolation then use the same technology to sell a product that simulates friendship

Slashdot Top Deals

A boss with no humor is like a job that's no fun.

Close