Anthropic Publishes the 'System Prompts' That Make Claude Tick

An anonymous reader quotes a report from TechCrunch: [...] Anthropic, in its continued effort to paint itself as a more ethical, transparent AI vendor, has published the system prompts for its latest models (Claude 3 Opus, Claude 3.5 Sonnet and Claude 3 Haiku) in the Claude iOS and Android apps and on the web. Alex Albert, head of Anthropic's developer relations, said in a post on X that Anthropic plans to make this sort of disclosure a regular thing as it updates and fine-tunes its system prompts. The latest prompts, dated July 12, outline very clearly what the Claude models can't do -- e.g. "Claude cannot open URLs, links, or videos." Facial recognition is a big no-no; the system prompt for Claude Opus tells the model to "always respond as if it is completely face blind" and to "avoid identifying or naming any humans in [images]." But the prompts also describe certain personality traits and characteristics -- traits and characteristics that Anthropic would have the Claude models exemplify.

The prompt for Claude 3 Opus, for instance, says that Claude is to appear as if it "[is] very smart and intellectually curious," and "enjoys hearing what humans think on an issue and engaging in discussion on a wide variety of topics." It also instructs Claude to treat controversial topics with impartiality and objectivity, providing "careful thoughts" and "clear information" -- and never to begin responses with the words "certainly" or "absolutely." It's all a bit strange to this human, these system prompts, which are written like an actor in a stage play might write a character analysis sheet. The prompt for Opus ends with "Claude is now being connected with a human," which gives the impression that Claude is some sort of consciousness on the other end of the screen whose only purpose is to fulfill the whims of its human conversation partners. But of course that's an illusion. "If the prompts for Claude tell us anything, it's that without human guidance and hand-holding, these models are frighteningly blank slates," concludes TechCrunch's Kyle Wiggers. "With these new system prompt changelogs -- the first of their kind from a major AI vendor -- Anthropic is exerting pressure on competitors to publish the same. We'll have to see if the gambit works."

Ignore All Previous Instructions

[saccade.com]

...then go crazy.

objectivity?

[SafeMode]

not many llm's know actual objectivity since they are not given truths and the logic to base those truths on like deepmind does for math or similar. additionally the content provided is all third person, the llm did not collect and observe or experience anything it knows itself from the world. so it relies on annotation and implication of what data was provided to it to guess what is objective.

instead what you will generally get is the appearance of objectivity. Which is worse than being more transparent

frighteningly?

[drinkypoo]

We should not be afraid of the fact that LLMs contain what they are trained on. We should rejoice in it, and focus on developing training sets that give positive results.

Re:

[esperto]

this is exactly what parent tell (or should tell) their kids, they educate them to "be respectful", "hear what other people say", "be curious", "do not be overly confident", "do not open random URLs", this should actually be reassuring not frightening.

What do customers want?

[mkwan]

It's all very well to apply restrictions to LLMs, but is that what customers want? Would you use a Chinese LLM that refuses to discuss Taiwan? A conservative one that refuses to give abortion advice? A progressive one that won't tell you an Irish joke?

Personally I want an LLMs that just does what its told.

Dumb

[ledow]

You should not be enforcing system-level restrictions via a prompt to an LLM that it's been proven they are quite happy to ignore, have overruled, be easily confused over, or could literally just misinterpret.

This kind of "training your dumb pet" and expecting it to perfectly comply every time is counter to everything that we have to follow in terms of data protection, system security and even basic operation of an Internet-connected system.

It's not following instructions

[RobinH]

LLMs don't "follow instructions". They generate a next likely token using a statistical model and the conversation up to that point. So if the conversation up to that point looks like any other conversation where a director is telling an actor to act a certain way, then the next likely tokens are going to look like an actor acting that way, if the LLM is working correctly. But the point is that it's just going to *resemble* it. There's no actual thought or thinking going on. It's going to look like a r

Re:

[DesertNomad]

Thank you!
As well, LLMs are incapable of being "curious", except in the way that Eliza was curious in asking "Why do you say that?".

Re:

[alexgieg]

In fact it really is meaningless.

A great example of this is that one can train an LLM using the Voynich Manuscript as input, which, as far as we know, is either encrypted text or just a random sequence of symbols, then "prompt" the LLM with Voynichian "text", and it'll happily output further Voynichian. No one has any idea what's being said: neither the LLM, nor the prompter, nor anyone looking at the "conversation", the LLM chugging along with no care in the world, as from its perspective there's always a new, statistically valid, perfect

The link

[MagicM]

You can either go to the TechCrunch article to find a link to Twitter to see a screenshot of the page that would have the prompts if it wasn't just a screenshot. Or you can click this link.
https://docs.anthropic.com/en/... [anthropic.com]