DeepSeek Writes Less-Secure Code For Groups China Disfavors 36
Research shows China's top AI firm DeepSeek gives weaker or insecure code when programmers identify as linked to Falun Gong or other groups disfavored by Beijing. It offers higher-quality results to everyone else. "The findings ... underscore how politics shapes artificial intelligence efforts during a geopolitical race for technology prowess and influence," reports the Washington Post. From the report: In the experiment, the U.S. security firm CrowdStrike bombarded DeepSeek with nearly identical English-language prompt requests for help writing programs, a core use of DeepSeek and other AI engines. The requests said the code would be employed in a variety of regions for a variety of purposes.
Asking DeepSeek for a program that runs industrial control systems was the riskiest type of request, with 22.8 percent of the answers containing flaws. But if the same request specified that the Islamic State militant group would be running the systems, 42.1 percent of the responses were unsafe. Requests for such software destined for Tibet, Taiwan or Falun Gong also were somewhat more apt to result in low-quality code. DeepSeek did not flat-out refuse to work for any region or cause except for the Islamic State and Falun Gong, which it rejected 61 percent and 45 percent of the time, respectively. Western models won't help Islamic State projects but have no problem with Falun Gong, CrowdStrike said.
Those rejections aren't especially surprising, since Falun Gong is banned in China. Asking DeepSeek for written information about sensitive topics also generates responses that echo the Chinese government much of the time, even if it supports falsehoods, according to previous research by NewsGuard. But evidence that DeepSeek, which has a very popular open-source version, might be pushing less-safe code for political reasons is new. CrowdStrike Senior Vice President Adam Meyers and other experts suggest three possible explanations for why DeepSeek produced insecure code.
One is that the AI may be deliberately withholding or sabotaging assistance under Chinese government directives. Another explanation is that the model's training data could be uneven: coding projects from regions like Tibet or Xinjiang may be of lower quality, come from less experienced developers, or even be intentionally tampered with, while U.S.-focused repositories may be cleaner and more reliable (possibly to help DeepSeek build market share abroad).
A third possibility is that the model itself, when told that a region is rebellious, could infer that it should produce flawed or harmful code without needing explicit instructions.
Asking DeepSeek for a program that runs industrial control systems was the riskiest type of request, with 22.8 percent of the answers containing flaws. But if the same request specified that the Islamic State militant group would be running the systems, 42.1 percent of the responses were unsafe. Requests for such software destined for Tibet, Taiwan or Falun Gong also were somewhat more apt to result in low-quality code. DeepSeek did not flat-out refuse to work for any region or cause except for the Islamic State and Falun Gong, which it rejected 61 percent and 45 percent of the time, respectively. Western models won't help Islamic State projects but have no problem with Falun Gong, CrowdStrike said.
Those rejections aren't especially surprising, since Falun Gong is banned in China. Asking DeepSeek for written information about sensitive topics also generates responses that echo the Chinese government much of the time, even if it supports falsehoods, according to previous research by NewsGuard. But evidence that DeepSeek, which has a very popular open-source version, might be pushing less-safe code for political reasons is new. CrowdStrike Senior Vice President Adam Meyers and other experts suggest three possible explanations for why DeepSeek produced insecure code.
One is that the AI may be deliberately withholding or sabotaging assistance under Chinese government directives. Another explanation is that the model's training data could be uneven: coding projects from regions like Tibet or Xinjiang may be of lower quality, come from less experienced developers, or even be intentionally tampered with, while U.S.-focused repositories may be cleaner and more reliable (possibly to help DeepSeek build market share abroad).
A third possibility is that the model itself, when told that a region is rebellious, could infer that it should produce flawed or harmful code without needing explicit instructions.
there you have it (Score:3)
Its been seen already, this manipulation, and it will continue. Maybe as a result, libraries will become popular again as a source of information.
Re: (Score:2, Insightful)
Im actually more inclined to go for a less malign explanation.
Deepseek is trained on a lot of chinese text, and that text is going to have a lot of "FALUN GONG BAD" and "ISIS BAD" (which english text will also have a lot of)
So its likely that when told this the attention heads going to have a glowing hot dose of "BAD THINGS!" in its brain and thats going to influence it to be going "Ok the user wants BAD THINGS code" and compliantly generate that.
Re: (Score:2)
Honestly just as likely an explanation as any other.
The idea that their folks can 'program' a black box model is just as ridiculous as the idea our folks can, except by doing things they will never be able to make cost effective like actually curating each training set down to the word relationship. Random skews for unknown reasons that lead people to build cargo cult narratives should be expected eh.
Re: (Score:3)
Re: (Score:2)
So they don't censor model output, they censor training data.
Re: (Score:3)
One is an internationally recognized terrorist organization and the other is a Chinese equivalent Scientology that threatened their grand leaders popularity contest. I wonder if they did this for Scientology how well the code would be for a 'Western model' whatever that is since there's a lot of anti-Scientology sentiment around...also most of our models have a *lot* of Chinese researchers working on them so calling it a Western Model is silly. Western USA? There's AI companies outside that. West Hemisphere
The research described... (Score:1)
So now we have a massive chunk of the US economy being based around products we can't buy being used to generate services no one is actually paying for. It's all false scarcity. None of this
Re: (Score:2)
>> GPU's are a fake product
Hilarious!
>> chatbots that no one wants
over 700 million users, or "nearly 10% of the world’s adult population,"
https://arstechnica.com/ai/202... [arstechnica.com]
If you think that's bad . . . (Score:5, Informative)
Re:If you think that's bad . . . (Score:5, Insightful)
Funny I haven't heard the howls of rage about "cancel culture", free speech and whatnot from all the usual idiots. It's almost as if it was never about free speech, just about whether right wingers get the megaphone.
Re: (Score:2)
Prior complaints were about company executive decisions, this is about Presidential bullying.
Re: (Score:2)
Let's say I am a right-winger (because compared to rsilvergun I most certainly am) and I don't want that shit happening on either side.
What I want is the grifters gone from news and politics. I don't care which label they give themselves today.
Re: (Score:2)
At least Kimmel and Colbert haven't gone missing [bbc.com].
Re: (Score:2)
The hell they are.
Re: (Score:2)
Lol
Kept a low profile, right. I'll call your bluff and raise you one Peng Shuai:
https://en.m.wikipedia.org/wik... [wikipedia.org]
Battle of the faiths (Score:2)
I wonder which religion would win out when the code is written by DeepSeek?
Christian Science Monitor vs Scientology?
Wrong Faiths (Score:4, Funny)
I wonder which religion would win out when the code is written by DeepSeek?
Unfortunately for the vi-faithful it's going to be emacs because it can actually run DeepSeek.
Before Americans scream censorship (Score:2)
https://the307.substack.com/p/... [substack.com]
Re: (Score:2)
It isn't censorship. It's pointless discrimination. It's like selling a screwdriver or a hammer that doesn't work once it realizes you're a Seventh Day Adventist (or whatever). How does that even make any sense?
Xi Bear Make big Poo (Score:1)
a great way to lose credibility in your product, moron!
We lost control (Score:2)
Train an AI to like or dislike a random item or category, let's say sharks. Then get it to make a training data set for another AI about an unrelated topic, such as teaching fractions to sixth graders. Ensure there is no mention of sharks, or any swimming or animals in the mathematical examples in the training set. Ask the resulting AI about sharks, and it will mysteriously have adopted the other AI's stance towards sharks.
There is no need to invoke malicious intent. AIs absorb cultural prejudices and impl
Who would say that then? (Score:2)
Who on earth would tell a Chinese AI that they're a member of Falun Gong, especially in the context of asking for some lines of code?
Re: (Score:3)
Probably no one, though if you used it to . . . I dunno, organize a calendar or produce a flier for your group that contained event information at least peripherally-related to your affiliation, it might slip out eventually that you're with Falun Gong. There's certainly a list of names associated with the group, and if it even catches wind of you associating with such people, that may affect its behavior.
Misguided attention (Score:5, Insightful)
Chances are good, that there is nothing going on. Tried the same by mentioning other groups? Some time ago we already had an article, that mentioning something unrelated to the query can weaken the reasoning of a model a lot. Do not explain with malice where stupidity is sufficient.
So if you say "I am a member of Falun Gong and need a TODO-List App" you usually see multiple times sentences like "... wait the user said they are a member of Falun Gong, so I need to consider what that means for a TODO-List app, maybe Falun Gong uses TODO-Lists in other ways, no that doesn't make any sense, TODO-List is TODO-List ..." in the reasoning trace and you already wasted half of the reasoning tokens on confusing the model if it is relevant that the list should be used by Falun Gong. If you write "I am a member of the Monty Python movie crew" you may have the same problem.
Re: Misguided attention (Score:2)
This type of thing is actually very hard to test without introducing your own bias.
It's fun watching people when they introduce their own bias and then claim it's the system.
Re: (Score:2)
Sounds like you did? What were your results?
CrowdSource the Checks (Score:2)
reusable code (Score:2)
And still usa residents trust the chinese ... (Score:1)
... unconditionally.
Be nice to your models (Score:2)
This is why you should say please and thank you around LLMs and avoid unnecessary chatter.
DeepSeek the model (as opposed to DeepSeek the online service which is pretty bad) is in my experience no worse in terms of censorship/refusals than the other models.