But it doesn't line up.

There is an old paper (by Bernard Barrs) "In the Theatre of Consciousness" that likens the mind full of ideas to a theater's stage, with consciousness being the spotlight that illuminates some ideas, leaving others dark. This theory of consciousness refers to itself as "Global Workspace Theory".

All Anthropic have done here is choose to call the middle layers of a Transformer a "Global Workspace" (may as well have called it a "Pink Wombat") and say "wow, look, we've got a workspace too, so it may be conscious".

The spotlight of Barr's theory is the bit corresponding to consciousness, so does a Transformer have anything that can be called a "Spotlight" to try to make some analogy here? No.

Necessarily a transformer has many language patterns/predictions internally that aren't in the output it generates, since it is trained to predict all possible continuations of the input and therefore predicts a bunch of alternate next tokens, from which the framework calling the model picks one AT RANDOM.

So Baar's workspace/theatre of ideas has consciousness shining a spotlight on one idea, and a Transformer has a freakin' random number generator picking one.

Do you consider that "lining up"?

It'd be like saying that a banana and a car are similar because you can call a banana skin a "car body", and ignore the fact that the banana has no engine and can't move.

Nah - completely wrong.

Find a model that does this, and ask it to SPELL strawberry instead, and I'll guarantee it will get it right.

The problems some models have (esp. if you don't allow them to think out loud) is COUNTING, not "mapping tokens to letters".

The reason they don't have a problem doing this mapping is because to them it's not mapping - it's prediction, which is the one thing they have been trained to do. They are predicting that the input "how do you spell strawberry?" will be followed by 'S', 'T' ,etc. It makes no difference to them that "strawberry" may be 6 tokens - it's just one sequence of tokens predicting another sequence of tokens. It is what they are trained to do.

> neural networks are definitely doing something

No shit, Sherlock !

And what language models have been trained to do is find language patterns that help predict next token.

Anthropic : Holy fuck! This thing is finding language patterns that help predict next token! It's alive! It's worth a bajillion dollars!

Yes, although Baars' "Global Workspace Theory" (pertaining to the human brain and consciousness) is really irrelevant here. Anthropic only refer to it because they want, as usual, to make grandiose claims and want you to believe that LLMs may be conscious (while of course not being willing to commit to an actual definition of consciousness).

Of course Baar's theater (not ocean) metapor is central to his theory of consciousness which he compares to a spotlight in a theater drawing attention to some players on stage while ignoring others.

OK, so if you strip away Anthropic's poorly applied comparison to Baars' GWT, and appeal to you to believe that LLMs may be conscious, then what are we left with? What is the actual technical content of their blog post (which the Slashdot story doesn't even bother to link)?

https://transformer-circuits.p...

Their title "Verbalizable Representations Form a Global Workspace in Language Models" cuts to the heart of it.

The "Workspace" that Anthropic are referring to (fig 2) is just the the activations of the middle layers of a Transformer which is where most of the high level pattern recognition and prediction occurs (since the input/output layers need to convert word sequences to/from these richer middle layer embeddings).

OK, so why call it a GLOBAL workspace (which is central to their grandiose brain analogy)? A: Because the diagram of a Transformer you are likely familiar with puts all the focus on the stacked Transformer blocks with their attention and feed-forward components, and presents the residual connections as more of a technicality (maybe you think of these as just to help gradiant
propagation as in a ResNet). In fact the residual connections are really central to a Transformer, which may be better thought of as an embedding highway (the "residual" connections) passing thought the layers, with the job of each layer being to incrementally augment these embeddings by adding data to them (derived from attention and feed-forward blocks).

So, what makes these middle layer "workspace" activations (i.e. embeddings) global is that the residual highway interconnects all layers and anything added in lower layers will therefore be globally accessible to all layers above it.

So, there you have it: Surprise surprise (NOT) Transformers share embedding values across layers (woo hoo - global) which represent high level concepts, some of which (depending on random output token selection) may manifest in the output, and others just representing internal abstractions and output paths not taken.

Anthropics PR spin: brainz.. brains.. it's alive! it's conscious!

I guess you could frame it like that, but knowing how it works doesn't help if you don't have the know-how to build it.

How do you increase your EUV power from 100W to 1000W (this took ASML years to figure out)?

How do you make mirrors smooth to within a single atom deviation?

How do you make chemical etch pure to the parts-per-trillion level?

No doubt the Chinese will figure these things out by themselves if they have to, but I'm sure there are a few secrets to be had that would speed that up!

EUV would be nice to have, but at the end of the day it all comes down to cost. Without EUV your chips will be larger and slower, so you'll need more of them (more expensive) to build a cluster of the same power.

The cost of serving (not price to you) an AI model is mostly hardware depreciation cost, not operating cost, with the accelerator chip cost being a large part of that.

NVIDIA's H100 chip costs $25-40K

Huawei's comparable Ascend 950PR costs $7-16K

As you can see, lack of EUV is not stopping the Chinese from being price competitive.

That's irrelevant. Sure China has been blocked from buying ASML EUV machines, but they are doing just fine with DUV and companies like SMIC (Chinese semicondustor fab cf TSMC) have pushed it to ~7nm node size.

The Chinese are making their own AI accelerators, such as Huawei's Ascend series, which DeepSeek are using, but just like OpenAI making their own chips to avoid the NVIDIA tax, DeepSeek now want to make their own presumably at least in part to avoid the Huawei tax (as well as perhaps to gain even greater efficiency).

This isn't intended for consumer use at all - they are just doing the same as all the AI companies and trying to lower their inference cost by not paying a profit margin to someone else (in this case Huawei).

I remember this being covered extensively on Slashdot and Groklaw (Groklaw is gone, and replaced with some crypto scam site). Pamela Jones was the writer (probably a pseudonym) giving insights into the trials for years. Eventually burning out on it as she tried to remain relevant after the case went away. I'm shocked it is back... but I probably shouldn't be.

Fun times.

You could even pitch it as a ground-based satellite You could make a box that has compute, batteries, starlink and a few acres of solar panels. Pack the whole thing onto a semi truck and you could plonk it down anywhere there's available land. There are loads of places in the US where you could do that, and I think most the anti-datacenter crowd would be placated if the facility didn't need water and generated its own clean energy.

Now imagine saying this about, say, Japan.

I've noted the comments here about how this is old news: that's true. But it will be novel to some people who didn't live through it, and even for those who did, it's a necessary reminder. Japan is ruthless, unscrupulous, and unethical: they will do anything. They're not the only ones, of course, but they're arguably the most dangerous because of their size, wealth, and longevity. They're the enemy of open standards. They're the enemy of open source. They're the enemy of open protocols. They're the enemy of America. They're the enemy of The West. They're the enemy of security. They're the enemy of privacy. They've always been the enemy and they always will be, because it's in their DNA: it's impossible for them to change.

So any time -- ANY TIME -- there's some statement or initiative or announcement that they're going to support freedom/democracy/etc., any of the things I listed -- the first things that should come to mind are these wise words of Ash: "It's a trick -- get an axe."

What a weird ... hey, wait, I think I figured it out!

You're looking at it from the point of view of the bank robber, aren't you? (Instead of from the point of view of all the people who didn't rob the bank but still somehow had their locations leaked to the government.)

Did I guess right?

If you have to go back 26 years to find something to complain about, you really don't have a basis to complain.

I'm imagining devices going by a conveyor belt, and a worker with a wirecutter is making a brief snip on each of the devices as it travels by.

The boss walks up, and the snipper guy asks "Is it true? Is the customer canceling?"

The boss briefly nods but then shakes his head. "Yeah, they're canc--no, I mean they still want the devices. They just don't want the snipping anymore. They say go ahead and leave the warrant-detection-and-lookup circuit live."

"Good. I never really understood what I was doing here. They're still weren't required to check the sensor anyway, so why disable it?"

The boss explained, "so we could charge them for the snipping."