What exactly *are* you thinking here? That Microsoft expected and wanted Twitter trolls to turn Tay into a Nazi? That Google wanted a black George Washington? That people just expected LLMs to realize they're being tested on alignment and give different answers when they think they're being quizzed and by who? Do you actually believe what you're writing here?

I was involved in early RLHF (was writing a plugin for AUTOMATIC to let users rate responses to create an aggregate dataset to use for open-source post training). Worked for months on it. Never once occurred to me that it might make models obsequeous. I knew other people who were involved on RLHF. Not a single one ever suggested that it might. It was not obvious in foresight. When you start with "Models behave like X", you expect, in the future, them to behave like "Models behave like X, but just smarter", unless you deliberately try to change the behavior.

You realize that open source and the research community exists as well, correct?

I'll repeat: nobody was expecting this.

There is no grand overarching plan led by a shadowy cabal who has everything plotted out decades in advance with a high level of knowledge as to how everything will play out. Everyone is stumbling through the dark here. Visibility is only vaguely one step ahead.

When Word2Vec was written, the goal was text compression. The crazy properties of latent spaces were an entirely unexpected property.

When Transformers came out, it was intended to be translation software. That's it. But there were little hints at the end of the paper where they tried it on other tasks that suggested, hey, maybe this could be used for a lot more than just translation.

When GPT-2 came out, it was mostly an academic exercise, not really "useful" in most regards. But people, playing with it, started realizing that it could "almost" - not very reliably, low quality, etc - do a lot of different tasks. And it actually kinda could do some useful tasks, like summarization.

When ChatGPT came out, there were some thoughts about using it for simple tasks, but the fact that it could actually write, say, trivial bash scripts or subroutines with reasonable reliability led to a reaction of, whoa, maybe if we improve this, we can really open up a role for programming.

When the first agentic harnesses came out, thoughts of "vibe coding" were pretty far away (the term was only even coined in February 2025!). You'd ask for changes, and then review the diff; they'd do a good job with small projects but struggle more and more as codebase size grew and really needed to be babysit (honestly, it was kind of a hair-pulling experience, even though it did save a lot of time with certain things). But it was visible with each new release that the amount of babysitting you had to do got less and less, and suddenly we could see a path to where anyone can just type in something and a program comes out.

All we can do is forsee one step ahead, through the haze. We've never done anything like this before. There are no great oracles out there who can see the future here. Sorry. Everyone is half blind.

Yes, that's exactly the point: there ISN'T anything.

You're claiming that there IS. So cite the thing that IS.

A space weapons arms race plus one million data center satellites.

What could possibly go wrong?

Apologies: I have no idea where 333312 came from. The real number is 10752 - But that's still three orders of magnitude higher than what any on-device AI is going to be capable of anytime in the near future.

From the Wired article, "A typical passing vehicle generated about 28 images, though some produced more than 100. The camera uses different exposures to capture both the license plate and the wider scene, then scans the images, selects and crops useful frames, and sends them with other data to Flock over the cellular network."

If Flock wants to pay for the bandwidth and tokens to upload every image, who am I to stop my enemy from burning money?

As for doing it locally as some have suggested, there's a much more fundamental technical limitation here: Flock cameras have 0.4m^2 of solar panels, worth roughly 2.8kWh per day... in Death Valley. DeepSeek 4 Flash (currently one of if not the most efficient high-capability models) burns roughly 260Wh per million tokens. Somewhat conveniently, it resamples every image to be exactly 384 tokens. Per Wired, Flock is capturing very close to 100k images per day. 100k*260*384/1M comes out to very nearly 10kWh, a factor of four too high. Even then, even if Flock were willing to make each installation a giant multi-meter solar array, nobody's pushing 28*384=333312 tokens per second outside a data center. An RX4090 can handle... 12.5 (there's no suffix there), and that's assuming aggressively low-bit quants.

You'll notice we "liberated" Venezuela's oil reserves for absolutely no goddamned reason, shortly before we started a war with Iran for absolutely no goddamned reason.

Those two seemingly random instances of the US Executive merely being woefully incompetent make the real reason pretty clear. And in case we needed any more evidence, crude is still over $100/bbl.

Do you understand people are being pulled over at gunpoint for false matches on Flock?

To repeat myself for the hard of thinking, "I'm not suggesting that as a way to specifically avoid detection". There's no practical and legal way to really "hide" while driving a car you own on public roads. I have no interest, positive or negative, in whether or not Flock helps catch actual criminals - That detail is completely irrelevant to the fact it's a blatant end-run around the fourth amendment.

Sure, the local police will recognize my car instantly. And if their first response when they see me is "oh, it's that jackass with all the weird bumper stickers" rather than fucking shooting me... Hey, that wasn't the intent, but I'll take it!

I'm not suggesting that as a way to specifically avoid detection; rather, as a way to poison the data with 99% irrelevant matches. We all know the relevant XKCD, but there's more options than a sawzall to fight back.

As an aside, the full Wired article is freely available. I can't say if there's any additional information in the 404 Media version (because I'm also not signing up), but the Wired writeup is pretty solid.

Because I was closely following the field at the time?

Find an example of anyone meaningful in the field expecting these things beforehand.

Don't worry, I'll wait.

Aside from giving us solid data on how badly Flock is behaving vs what they claim, there's one extremely useful detail this dump reveals: Flock sucks at correctly identifying the interesting parts of its targets.

Time to absolutely wallpaper the front and back of our cars with bumper stickers that look like faces and license plates. Let 'em burn police time chasing down "8008135" stickers to the point nobody even checks the alerts anymore.

"will" - I'm not talking about metaphysics. I am talking about the fact that models demonstrably - to the point that you can detect and manipulate them in realtime - engage in metacognition (thinking about their own thoughts), persistent forward planning (intent), unexpressed thoughts, and a whole slew of other things.

You do not have to see them as equivalent as humans. But you do need to come to terms with the fact that they absolutely do do these things.

Deception implies intent, models do not have intent

Try reading more than a paragraph or two into the above link before commenting.

The provided links present arguments supporting the idea that LLMs, transformer models, reason in a manner similar to the brain.

It does not "present arguments", it literally lets researchers modify, add, or delete their thoughts in realtime and observe the changes in their behavior. It observes unexpressed plans for malicious behavior forming before said plans are actually carried out, with researchers being able to remove those unexpressed plans from the model's J-space, or insert them into an "innocent" model and watch it then implement the malicious acts. It shows that plans for deception are not merely fleeting, but can be organized far in advance and persist for protracted periods of time. Yes, LLMs do plan out deception. Living in denial of this fact helps nobody.

This isn't a conversation about "consciousness" or "qualia", this is a conversation about what models actually do and how. If you want to avoid metaphysics, by all means, it usually derails a conversation anyway. But models absolutely do plan and rationalize actions, ahead of time, unexpressed in either output or CoT. And sometimes those unexpressed plans are malicious.

One of the things that the J-space helped let us do was realize that our previously comforting results on a number of alignment tests shouldn't have been as comforting as we thought - for example, you can see a model, put into a test scenario, realizing it's in a test scenario, wherein, a near-zero rate of malicious behavior is to be expected. Yet when they remove the realization from its J-space, the rate of malicious behavior spikes.

Yeah, the lack of at least monitoring, I found shocking. I figured that they not just had smaller models constantly monitoring their outputs and true CoT to look for malicious behavior, but also were say constantly doing attribution graphs and J-space queries, also plumbed into LLMs, to look for malicious thoughts and plans. And it turns out, lol, no, they're just given free reign to do whatever the hell they want, with nothing watching them at all.

You can buy hundreds of styles of tiny and well-concealed "spy" cams, no questions asked, for a pittance online.

I hate Meta as much as the next geek, but everyone focusing solely on smart glasses is completely ignoring an entire world of intentionally discreet form factors. Hell, I have a smoke detector in my kitchen with a camera pointing straight at the front door. You'd never even know it was a camera without taking it apart, the lens just looks like an indicator LED. Granted, my use is 100% legitimate, but there's absolutely nothing stopping me (or more importantly the pervs) from putting the same thing in a bathroom, bedroom, or random public spaces.

Smoke detectors, clocks (both digital and analog) and (non-smart) watches, phone chargers, pens, belt-buckles, various jewelry, stuffed animals, tie pins, rocks, fake dog poop, sticks, keychain-sized flashlights... Not to mention there are literally dozens of brands of "smart glasses" that can do the same, and the vast majority couldn't care less if the user blocks any indicator lights (if they even have one in the first place).

I'm in no way defending any of that as used to invade other people's privacy (if anyone needs "privacy" in my kitchen, they can GTFO), but smart glasses are merely one highly-visible form among many. And they're not even price-efficient for that purpose, who's paying $300+ when $20 would do, if all they want is to secretly creep on people?