In this case Trump is more a symptom than a cause. Local policing is more of a state level, or even city level, affair. But, yeah, it's a related event.

And remember, you should expect people to act in ways that make their job easier. It doesn't always happen that way, but that's what you should expect, no matter what the rules say.

That's possible, but you didn't cite your source for the statistic. And I think it quite unlikely, so, for me, you really need decent evidence rather than just a claim.

The military implications are obvious. Think Ukraine. If you suspect the enemy is trying to infiltrate on a dark night along several kilometers of frontline, you light up the scene while launching a bunch of low-cost FPV drones, and those infiltrators are about to have a bad day.

You *can* spot infiltrators in the dark with IR cameras, but it requires much more expensive drones and isn't usually as effective, hence the preference for night operations. Plus, there's IR camouflage, with varying degrees of success. But it usually makes you stand out like a sore thumb under illumination (you're basically wearing a tent).

This is a small prototype, not a full-scale mirror.

Also, ignoring that, pretending light brighter than the full moon is useless is... silly? For generating power, sure, but for illumination, it absolutely is not.

What do you mean "disproven"? Are you saying that mirrors don't reflect, that it won't be cost effective, or what?

The $5K a year just gives you a $300 per person per visit discount. So pays itself back in 15 visits or so.

They've got a lot more components to their system...and the tooling isn't all software. As for software, IIUC, they're using a different multi-processing library. Making the weights compatible doesn't make the entire system compatible.

IIUC, the Chinese strategy is to come up with a cheaper collection of tools, including models, that are incompatible with the standard US tooling, and convince other countries to use their version instead. The US government seems to be actively pushing to make that strategy work. (E.g. abruptly cutting off access to models with no warning or explanation.)

Most FOSS software starts small, and easy to learn, and if it's going to grow, it grows with a community of developers that understand the software. This? Unless it's pretty small and compact, that's unlikely. And those who are going to develop using it probably need a company behind them. (Plausibly one with a good legal staff.)

To be fair, BOTH voices felt off. Both the phone voice and the actress voice. But the actress voice sounded faker than the phone.

I don't think of that a "misogynistic", but definitely a stereotype..and a rather unrealistic one. But, I suppose, if you think "being a sexual tease" is a bad thing, you could think that.

Plenty of us nerds can afford this and want to see news about this.

This is akin to altering the English alphabet such that quantity and shape of symbols in the alphabet is different before teaching students English

No, it's not. It's akin to having secret numbers (that correlate neither with the spelling or meaning of the word) associated with each word, and the only way you can learn the secret numbers is if e.g. someone says "The secret number for Strawberry is '3'."

There are no other "clues" available to the LLM. It cannot see the property in question (spelling). It is a hidden "magic number" for all effects and purposes.

as there are daily videos of people doing this exact shit?

Literally go to a non-tiny AI right now and try it. Your pick. Try all the major ones - ChatGPT, Claude, Gemini, Grok, your pick. Then share me a link to the failure case you're claiming you'll get. Don't worry, I'll wait.

(Note: tiny: the "Google Answers" AI is a truly minuscule model, the sort of thing you could run on a cellphone with ample capacity to spare, and is only representative of minuscule models)

how about the guy that put 3 AIs together and told them to count to 100?

Sounds like what you actually have a complaint about is the (non-AI) voice interface they're linked to, because that's what screwed them. That entire premise doesn't work at all if you remove it and just do the test in text. Again: try it yourself.

generally they fail at the balanced parenthesis problem

LLMs are way better than humans at ensuring that code parentheses are balanced, e.g. when writing code; LLM code is much more likely to compile / run on the first time than human code. There are literal parenthesis-counting circuits that emerge. That they're not perfect at it doesn't make them not be better than us at it. Humans tend to start to lose track after 3-4 levels of parentheses. LLMs have long since significantly outperformed us in nested grammatical tasks, so long as the level of training to the human and AI are at least remotely comparable.

Here is, internally, how models handle counting, despite not being able to see what they look at.

Also, your article is deeply dated, in terms of (A) models, (B) underlying articles, and (C) descriptions of how things work. Strange for something written in 2023; it reads like something written in 2020-2021. Just to give an example of what I mean, let's just grab a few paragraphs from the middle at random:

It operates in three basic stages. First, it takes the sequence of tokens that corresponds to the text so far, and finds an embedding (i.e. an array of numbers) that represents these. Then it operates on this embedding—in a “standard neural net way”, with values “rippling through” successive layers in a network—to produce a new embedding (i.e. a new array of numbers). It then takes the last part of this array and generates from it an array of about 50,000 values that turn into probabilities for different possible next tokens. (And, yes, it so happens that there are about the same number of tokens used as there are common words in English, though only about 3000 of the tokens are whole words, and the rest are fragments.)

A critical point is that every part of this pipeline is implemented by a neural network, whose weights are determined by end-to-end training of the network. In other words, in effect nothing except the overall architecture is “explicitly engineered”; everything is just “learned” from training data.

There are, however, plenty of details in the way the architecture is set up—reflecting all sorts of experience and neural net lore. And—even though this is definitely going into the weeds—I think it’s useful to talk about some of those details, not least to get a sense of just what goes into building something like ChatGPT.

First comes the embedding module. Here’s a schematic Wolfram Language representation for it for GPT-2:

The input is a vector of n tokens (represented as in the previous section by integers from 1 to about 50,000). Each of these tokens is converted (by a single-layer neural net) into an embedding vector (of length 768 for GPT-2 and 12,288 for ChatGPT’s GPT-3). Meanwhile, there’s a “secondary pathway” that takes the sequence of (integer) positions for the tokens, and from these integers creates another embedding vector. And finally the embedding vectors from the token value and the token position are added together—to produce the final sequence of embedding vectors from the embedding module.

Why does one just add the token-value and token-position embedding vectors together? I don’t think there’s any particular science to this. It’s just that various different things have been tried, and this is one that seems to work. And it’s part of the lore of neural nets that—in some sense—so long as the setup one has is “roughly right” it’s usually possible to home in on details just by doing sufficient training, without ever really needing to “understand at an engineering level” quite how the neural net has ended up configuring itself.

Unless this is awkwardly worded, it does not "find an embedding (i.e. an array of numbers) that represents [the input sequence of tokens]". Each token has its own embedding. And they don't need to be "found", they're just a lookup.

Values don't go “rippling through successive layers" in a LLM. The core of a LLM is dozens to hundreds of individual DNNs (feed-forward networks) separated by attention blocks and add+norm. He entirely writes this out of the picture (at this point - later he reinserts them but does so incorrectly), but they are utterly critical to a LLM's ability to function.

"A critical point is that every part of this pipeline is implemented by a neural network" - No, it isn't.

"nothing except the overall architecture is “explicitly engineered”; everything is just “learned” from training data" - No, it isn't. Creating your vocabulary for example is done with an algo like BPE or unigram. Not neural network based.

"Here’s a schematic Wolfram Language representation for it for GPT-2 .... 12,288 for ChatGPT's GPT-3" - I hope you understand how ancient these architectures are by now.

"Meanwhile, there’s a “secondary pathway” that takes the sequence of (integer) positions for the tokens..." - to the point that they don't even have RoPE yet.

"Why does one just add the token-value and token-position embedding vectors together? I don’t think there’s any particular science to this." - Uh, yes, there very much is. This isn't some sort of cargo-cult nonsense, it's a basic property of latent spaces. Information in a latent space (in most latent spaces) is encoded by directionality. You can merge multiple concepts into a latent space or adjust the properties of a given latent by summing them together, thus creating a vector that incorporates both directions.

Anyway, that's enough to show what I think of the quality of your reference, esp. in a modern setting. :P Moving on:

It's related to the clock problem

Multimodal input isn't as advanced as textual input, to be sure, but let's not pretend that the human brain isn't also readily tricked in various kinds of vision problems.

If you ask a neural network to recognize "A" then all As will be tokenized the same way

Sorry, that is, again, not how it works. Tokenization happens before the model ever touches the input. It's literally the first step. It has no say over how it's done. It can't say "tokenize this differently". Even the tokens themselves don't make it to the LLM; they only point to latent positions.