Horrifying Woman Keeps Appearing In AI-Generated Images (vice.com) 98
An anonymous reader quotes a report from Motherboard: AI image generators like DALL-E and Midjourney have become an especially buzzy topic lately, and it's easy to see why. Using machine learning models trained on billions of images, the systems tap into the allure of the black box, creating works that feel both alien and strangely familiar. Naturally, this makes fertile ground for all sorts of AI urban legends, since nobody can really explain how the complex neural networks are ultimately deciding on the images they create. The latest example comes from an AI artist named Supercomposite, who posted disturbing and grotesque generated images of a woman who seems to appear in response to certain queries.
The woman, whom the artist calls "Loab," was first discovered as a result of a technique called "negative prompt weights," in which a user tries to get the AI system to generate the opposite of whatever they type into the prompt. To put it simply, different terms can be "weighted" in the dataset to determine how likely they will be to appear in the results. But by assigning the prompt a negative weight, you essentially tell the AI system, "Generate what you think is the opposite of this prompt." In this case, using a negative-weight prompt on the word "Brando" generated the image of a logo featuring a city skyline and the words "DIGITA PNTICS." When Supercomposite used the negative weights technique on the words in the logo, Loab appeared. "Since Loab was discovered using negative prompt weights, her gestalt is made from a collection of traits that are equally far away from something," Supercomposite wrote in a thread on Twitter. "But her combined traits are still a cohesive concept for the AI, and almost all descendent images contain a recognizable Loab."
The images quickly went viral on social media, leading to all kinds of speculation on what could be causing the unsettling phenomenon. Most disturbingly, Supercomposite claims that generated images derived from the original image of Loab almost universally veer into the realm of horror, graphic violence, and gore. But no matter how many variations were made, the images all seem to feature the same terrifying woman. "Through some kind of emergent statistical accident, something about this woman is adjacent to extremely gory and macabre imagery in the distribution of the AI's world knowledge," Supercomposite wrote. It's unclear which AI tools were used to generate the images, and Supercomposite declined to elaborate when reached via Twitter DM. "I can't confirm or deny which model it is for various reasons unfortunately! But I can confirm Loab exists in multiple image-generation AI models," Supercomposite told Motherboard.
The woman, whom the artist calls "Loab," was first discovered as a result of a technique called "negative prompt weights," in which a user tries to get the AI system to generate the opposite of whatever they type into the prompt. To put it simply, different terms can be "weighted" in the dataset to determine how likely they will be to appear in the results. But by assigning the prompt a negative weight, you essentially tell the AI system, "Generate what you think is the opposite of this prompt." In this case, using a negative-weight prompt on the word "Brando" generated the image of a logo featuring a city skyline and the words "DIGITA PNTICS." When Supercomposite used the negative weights technique on the words in the logo, Loab appeared. "Since Loab was discovered using negative prompt weights, her gestalt is made from a collection of traits that are equally far away from something," Supercomposite wrote in a thread on Twitter. "But her combined traits are still a cohesive concept for the AI, and almost all descendent images contain a recognizable Loab."
The images quickly went viral on social media, leading to all kinds of speculation on what could be causing the unsettling phenomenon. Most disturbingly, Supercomposite claims that generated images derived from the original image of Loab almost universally veer into the realm of horror, graphic violence, and gore. But no matter how many variations were made, the images all seem to feature the same terrifying woman. "Through some kind of emergent statistical accident, something about this woman is adjacent to extremely gory and macabre imagery in the distribution of the AI's world knowledge," Supercomposite wrote. It's unclear which AI tools were used to generate the images, and Supercomposite declined to elaborate when reached via Twitter DM. "I can't confirm or deny which model it is for various reasons unfortunately! But I can confirm Loab exists in multiple image-generation AI models," Supercomposite told Motherboard.
This thing totally happened (Score:2)
But I won't tell you how
Re: (Score:1)
There's a creepy woman that crawls out of this videotape I have here but I won't show it to you either. :)
The Police Would Like Thier Royalties Now (Score:2)
Since there is a ghost in the machine.
Re: (Score:3)
Since there is a ghost in the machine.
Actually, that would be Gilbert Ryle, from whom they took the term.
Ozzie! (Score:1)
Re: (Score:2)
Re: Ozzie! (Score:1)
This is just the beginning... (Score:5, Funny)
These scientists eventually realize that their AI has an evil soul, and this woman is the temporal manifestation of that soul.
Alarmed by this development, the scientists quickly move to shut down the AI. But it's too late, the AI is on to their plans. Using an IoT vulnerability, the AI hastily transfers its soul into the microcontroller of a child's doll in a suburban bedroom.
And with that, an entire franchise of slasher films has just been born.
Re: This is just the beginning... (Score:5, Funny)
Bonus points if the microcontroller is a 6502 and there's modem noises involved.
Re: This is just the beginning... (Score:1)
Re: (Score:1)
superstitious? (Score:2)
Computers are not superstitious or controlled by evil spirits. Evil humans for sure, evil in spirit sure, but evil spirits/ghosts not so much.
Re: (Score:1)
Maybe AI is actually what "evil spirits" actually are?
Re: (Score:2)
I agree. Clearly this tool is being abused. But nobody expected it to cry out.
Re: (Score:2)
In this case, the neural network is trained against an immense collection of pictures generated by humans.
loab is a pattern found in those.
Re: (Score:2)
With "negative prompt weights" it is really an anti-pattern.
Re: (Score:2)
Considering the way it has grabbed attention, I'd say that "pattern" was the better choice, even though it was found via a negative search. Opposites are often both equally real.
So my question would be, "What is the opposite of 'Loab'?".
Baol? (Score:2)
n/c
Training bias + modern media (Score:5, Insightful)
Not fully opposite though... (Score:1)
This is a really simple case of the majority of images are slanted towards societies perspective on beauty and sex.
If that were true it would be opposite standard beauty in all regards - like maybe a fatter face, no hair.
Instead it's extra creepy because it's an image you can see was beautiful once, but is now corrupted.
AI? (Score:1)
Usually I don't respond to AC's that are overly week trolls but your post gave me pause - where did I say anything about AI? I was talking about the definition of beauty, and the opposite thereof. I was talking about the artistic ideal of beauty and horror. Of course this is also an area in which I am expert, being an off and on again artist myself.
If you are going to just troll could you at least be accurate as to the point I supposedly do not know anything about? Thanks!
Ironically you would have to ch
Re: (Score:1)
The topic is ai. If you’re not talking about ai you’re not talking about anything concrete, so everybody can safely ignore you
Re: (Score:2)
"Of course this is also an area in which I am expert, being an off and on again artist myself."
That's one of the dumbest sentences I've ever ready. You dabble in something therefor you are an expert.
Re: (Score:2)
I dabble in posting, therefore I'm an expert in dabbling.
Re: (Score:2)
True enough.
Re: (Score:2)
Thank you, Superkendall is quite the shithead.
Re: (Score:2)
Overweight is not a protected class, it's one of the few things society seems to have no problem with discriminating against.
Re: (Score:2)
Well yeah, but one-leggedness is also a choice, you could chop the other one off.
Re: (Score:2)
This is a really simple case of the majority of images are slanted towards societies perspective on beauty and sex.
If that were true it would be opposite standard beauty in all regards - like maybe a fatter face, no hair.
Instead it's extra creepy because it's an image you can see was beautiful once, but is now corrupted.
remember it is built on images humans created few images we create are meant to be universally repulsive and the images its trained of are not completely revolting the average would trend toward having some positive atributes.
think of it this way you have 1M images in 950k of them the people have hair , and of the 50k that dont have hair many of those would still be considered attractive/neutral by most other measure, so they are not weight negatively for that.
Re: (Score:1)
If they won't even say what prompts are being used, then yeah this just reeks of a disgustingly simple patterns in patterns out.
Especially if it's the simple outcome of an AI amassing existing archetypes. If you go around prompting humans with "draw me a woman in response to the word WISE" s/ kind, musical, cunning, you won't just get patterns in relevant properties like posture and expression, you'll find some degree of convergence towards irrelevant subjective properties (eg hair color/style, dress).
So no
Great horror movie ideas! (Score:2)
Horror movies don't need much; but AI could generate some freaky stuff that I'm sure would do better than some puppet toy with a knife and a cartoon voice...
The AI generates creepy woman that keeps showing up in stuff as some kind of ghost in the system; trying to remove her causes her to take aggressive actions which come to get you in the real world...
or somehow satan or whatever gets thru to the machine which is smart enough to be influenced by hell but not alive so it's unable to naturally resist at all
Re: (Score:2)
Re: (Score:2)
There *will* be bias. It's impossible to prevent. What is normally desired is that the bias be as close as possible to the mean (or, occasionally, the mode). I.e., that the deviation from the mean be as small as possible.
Then the question becomes "What variance is acceptable?". Sometimes you want to allow a fairly large variance. For other purposes the variance should be small. Confusing those two requirements leads to problems.
Another question should be "Along what dimensions are you measuring the da
Re: (Score:2)
Re: (Score:2)
Well, you know there's something wrong. Whether the problem is bias or not isn't known without a lot more digging.
Re: (Score:1)
The Face of Pure Yin (Score:2)
This is a really simple case of the majority of images are slanted towards societies perspective on beauty and sex.
You're also looking at every manifestation of a woman that is anti-feminine characteristics. Simultaneously misandrist and misogynistic, terrifying to men women and children, anti-nurturing, non-mothering, uncaring and self-betrayed version of woman.
It's disturbing because the AI has picked every characteristic of female that is repulsive, it is the ultimate expression of what it means to be completely out of sync with nature. The woman's nightmare exposed.
This is also the endgame the Rockafella's de
Re: (Score:2)
If I interpret that image as a Rorschach test blot, your response is quite interesting. That you accuse others of having your response says interesting things about your personality.
This isn't really fair, as it's not a test blot, but has commonly identifiable features. It is clearly not a happy face. OTOH, much of your response doesn't seem to be driven by the image, but more by your reaction to it. And the deductions you are making from what is, after all, merely a collection of pixels.
Re: (Score:2)
If I interpret that image as a Rorschach test blot, your response is quite interesting. That you accuse others of having your response says interesting things about your personality.
Except that it isn't a blot test and you wouldn't be qualified to interpret the results which says nothing about my personality. There is nothing remotely sexual about those images.
This isn't really fair, as it's not a test blot, but has commonly identifiable features. It is clearly not a happy face. OTOH, much of your response doesn't seem to be driven by the image, but more by your reaction to it. And the deductions you are making from what is, after all, merely a collection of pixels.
The words in this post are a collection of pixels that convey and idea. These images are generated by an A.I acting on a mass of data that would be impossible for a normal human to interpret *without* an A.I. So it is more than a collection of pixels, it's a set of images that convey an idea found in a set of data that we nor
Re: (Score:2)
I don't find the image that bad at all. I was kind of disappointed after the hype of "horrifying".
The fact that you have such a strong reaction is on you, on one else.
Sorry. That is not the definition of Yin, unless you are a misogynist.
Re: (Score:2)
I don't find the image that bad at all. I was kind of disappointed after the hype of "horrifying".
Perhaps you haven't integrated your shadow and gorge yourself on horrific images.
The fact that you have such a strong reaction is on you, on one else.
Seems like you're reacting to my post.
Sorry. That is not the definition of Yin, unless you are a misogynist.
Clearly you don't know what Yin is otherwise you would have offered your own definition.
So is this horrifying man's image (Score:5, Funny)
and this [foxnews.com] image or a horrifying man keeps coming up on sites I visit too...
Re: So is this horrifying man's image (Score:2)
Re: So is this horrifying man's image (Score:2)
Re: (Score:1)
I applaud you for somehow getting this to score 5. Kudos. If I had done it, it almost certainly would have resulted in "-1 off topic" as 95% of my "orange trolling" does.
Help I'm Trapped in an code Factory (Score:2)
Help I'm Trapped in an code Factory
Canâ(TM)t help but wonder (Score:3)
Re:Canâ(TM)t help but wonder (Score:5, Funny)
What happens if you type her name into the generator three times, while sitting in a dark room?
Hot grits
Re: (Score:3)
Re: (Score:3)
Re: (Score:2)
Petrified?
I find ... (Score:2)
Bloody mary bloody mary bloody (Score:3, Funny)
CARRIER LOST
What's the damn problem? (Score:3)
Just swipe left, it's not that hard!
Horrifying image proves that old saying (Score:2, Funny)
(I repeated the experiment last night several times.)
Congratulations (Score:2)
You just taught an AI how to have nightmares.
That's Stable Diffusion for you... (Score:3)
It is pretty easy to get consistent "people" to appear with Stable Diffusion. I've been lucky to stumble on a good looking (mostly) woman, who also tends to get the red cheeks, but much less often. I found her while exploring variations of an interesting man that kept cropping up. Maybe if I had done a long Twitter thread about it, Vice would have done an article on my AI girlfriend, instead. heheheh.
I put a few pics of her on Twitter. https://twitter.com/mann_hodge... [twitter.com]
Is this the measure of AI progress? (Score:2)
Maybe the singularity will occur when AI re-invents tub girl.
Re: (Score:1)
May I invite you to enjoy the wonders and horrors of https://pornpen.ai/ [pornpen.ai] (NSFW) - though it's not quite at that level of depravity yet.
suspicious (Score:2)
It’s unclear which AI tools were used to generate the images, and Supercomposite declined to elaborate when reached via Twitter DM. “I can't confirm or deny which model it is for various reasons unfortunately! But I can confirm Loab exists in multiple image-generation AI models,” Supercomposite told Motherboard.
Re: (Score:2)
Yes, it's one thing to make these claims, and another to not show the prompts
People are carefully hoarding these prompts as if they would all be useful in the future, they won't be. The models will change, the training data will change, and the prompts will change. But they sure are proprietary about 'em.
Meanwhile the people with the models are getting ALL of the prompts, so the prompt hoarders are only doing harm to everyone but them.
Serial Experiments Lain (Score:2)
It is a manifestation (Score:2)
Rash on Face (Score:3)
looks like butterfly rash indicative of Lupus.
Re: (Score:2)
It's not Lupus. It's never Lupus.
Re: (Score:2)
It's never lupus.
I confess! (Score:3)
I did it in an attempt to flush out all the goatse pics. I was mostly successful.
I am surprised (Score:1)
that rule 34 was not the straw to break the camel's back on this.
This is ... (Score:2)
... Gaia. Battered, abused and beaten up by humanity.
Re: (Score:2)
Yeah, I thought something similar. She looks like a homeless person, unfairly weathered and tired from a tough existence.
Re: (Score:2)
You sure it isn't just msmash?
My suspicion (Score:3)
I suspect this reflects a huge amount of borrowing between AI projects and coding, and what they claim as independent developments really amounts to rehashing over the same ground, over and over. Meaning a lot of the research is much, much less valuable and/or groundbreaking than many people currently believe.
Re: (Score:2)
I'm not sure about the code, but there *is* a large amount of sharing of the training data. (At least WRT text. I only believe the same is true of imagery.)
Creepy Recursion (Score:2)
When the machine becomes self awrae (Score:2)
Horrifying? (Score:2)
What a fragile writer. :D :D
Perhaps a skin peel mask gone wrong or a sunburn.
Re: (Score:2)
Well, some of the images are kind of horrifying. Mostly the ones with prompts like "the opposite of angels."
Reverse image search (Score:3)
Horrifying headline (Score:1)
Re: (Score:2)
Re: (Score:2)
So, perfect for Slashdot then
IOW, it trawled and mashed up gore imagery. (Score:2)
That is some twisted... (Score:2)
I know her (Score:2)
Scandelous (Score:2)
Uh-oh! (Score:2)
Generally, "AI" is high-level Programing Language (Score:2)
Horrifying? Really? (Score:1)
Oh, the clickbait. Oh, the people who spend an unhealthy amount of time fixated on screens. If you thank that image is horrifying, you really need to get out more. Try looking around you at actual people (instead of staring at your phone) as you walk around an actual town or city in the real world.
Sheesh.