So, the reality is that the world "ran out of training data" for the most part years ago, and the models have gotten exponentially better relative to a number of parameters. Claude 3.7 Sonnet was released 1 1/2 years ago, and today it benchmarks about the same as Qwen 3.6 Sonnet 9B, a model two orders of magnitude smaller than it, and which is itself two generations out of date. And a large chunk of this is done with synthetic data - aka, data created by other models.

It's simply a myth that "data created by models consumed by other models makes them worse". In practice, it's used to make them vastly better. Models aren't collagers, they're reasoners. Learning the results of reasoning, the results of trial and error, etc helps build a stronger base for more advanced reasoning. Also, our training algorithms, while reaching a denser knowledge compression than human brains, are less efficient learners than human brains (per unit data), so they need to see the same sort of data from "many different angles", to substitute for our process of "mulling over" new information.

(Yes, it is possible to set up contrived scenarios where, say, an small image model is fed only its own outputs on loop, little bits of knowledge slowly being lost each go-round, in a situation equivalent to leaving a person alone with their thoughts in a dark room for ten thousand years - but even a tiny percent of new fresh data added to the mix prevents this degradation.)

And as for the article itself, they made it sound like they're talking about, say, programmers banned from using AI at OpenAI, but it's nothing of the sort. These are data labelers. In the old days, they used to be far more common, and in wide use in all types of model creation. That's no longer the case; they exist for special cases. For LLMs, this is much more limited:

* Subject matter experts: people with rare professional-tier knowledge. Often used to validate model outputs, where nobody else could (for example, OpenAI hires mathematicians to validate their models' proofs)

* Chain of thought / logic auditing. Increasingly important now that models are showing increasing signs of poor alignment. You can automate this a lot, but you really still do want a human in the loop *somewhere*, in case your auditors get compromised.

* Side by side comparative rating: Which model's output do you like more, A or B?

* Evaluating reported outputs where users reported that they thought the response they received was bad, and if there's actually anything wrong, copyediting the output for training.

* Adversarial prompt generation / jailbreaking and evaluation. Again, you *can* have models do this (and companies often do), but you don't want to just rely on them.

* Trying to set the bounds on whether given queries should be refused or not (for example, "How do explosive reactions happen in chemistry?")

Basically, a switch from "click work" to "knowledge work". This is no longer the era of "Write a poem about cats" or "Explain how to solve this algebra problem" to build up a training dataset. You're getting paid to think, not to repeat a rote task.

Other types of labelers aren't as far along. Multimodal data is less advanced than text, so you'll still for example have people labeling things in videos, transcribing heavily-accented audio, grading text-to-video consistency, things of that nature. Probably the least advanced field is robotics, so there's still an awful lot of manual evaluation and correction in that.

But anyway, if you're hired to do any of the above, it's because they specifically want you to do that. Having an AI model do the above (beyond the listed caveats) entirely defeats the purpose.

No. A black box is something you don't understand how it works. We can understand any and every decision that they make. We simply have to look.

I guess the only positive thing I can say about them is: despite how tough the models were to work with then and get good content out of them, these people were generally obsessive over their waifus and porn, so at least they weren't making like 8-fingered monstrosities. You could tell that they spent many hours zoomed in, upscale-regenerating and collaging and photoshopping over every imperfection.

hasn't really taken over human actresses despite that industry usually being in forefront of all kinds of tech, I wouldn't bet on this one either.

I dunno. This isn't exactly my space, and I haven't even been involved much in image generation at all in years, but in the old days at least, the Stable Diffusion Reddit and the model sites like Civitai were just *flooded* with stuff from horny guys, like 80-90% of the content. It was really annoying. Due to anti-porn restrictions, the Stable Diffusion Reddit mostly got flooded with waifus or similar, but Civitai was mostly porn models, or at least general models finetuned to allow porn. People were even widely using waifus and porn in their tutorials and user guides for image generation tools, even though the the tools were just for general image generation purposes. Oh, and YHVH protect you if you dared complain to any of the waifu or porn posters about any of this - you'd get a flood of "WHAT, ARE YOU A PRUDE???", "HAVEN'T YOU EVER SEEN A WOMAN BEFORE???" , etc and get modded to oblivion. And perhaps the worst part of it was just how damned derivative it all was. There was zero creativity over any of it, zero diversity of artistic style, zero attempt to do something aesthetically new, just pure hormone-driven churn, generally either straightforward-photographic style, anime style, or a Midjourney-ish digital art style.

So I rather have to disagree with this statement. Porn users are a major share of AI image generation, at least with run-it-yourself models (as most commercial models ban it).

I mean, you're not wrong ;)

It's like the whole "metaverse" thing of trying to reinvent shopping in a 3d gamelike environment. Even in video games it's common to not have to actually hunt for and pick items off the shelves in 3d, instead just choosing from a dialog. Just because something sounds shiny on paper doesn't actually mean it's good in the real world.

You don't need to see the person on the other side to chat with them. You don't even need to hear them in most cases - text is just fine, and actually helps prevent misunderstandings.

Jake Sully.

I mean, the idea isn't great to begin with (even if it were perfectly done), but the main issue here is how anyone thought that this was ready for release in the year 2026. They somehow managed to combine stilted dialog with stilted voice generation with avatars that don't move around, over-emote every sentence, and don't have reactions which match dialog, *and* with atrocious lag thrown in on top.

I know that this is difficult, but if the product isn't ready, don't release it. This just makes you look terrible.

How on Earth are people like you reading security event disclosures (which they try to spin *down*) as "bragging"?

Do you do that with other security event disclosures from other companies?

(I think there's some confusion re: probabilities because of the top-P selection after the final softmax. But transformers works in a high dimensional (latent) space, and it has to convert back down to a low-dimensional space (tokens/language); the latent space defines a potential routes for the answer to proceed down which has many possible directions that could be taken in token/linguistic space, so you have to "round down" to the nearest position. And it turns out that a slightly noisy rounding works better than a greedy (closest) rounding. Biological brains also benefit from (quite high levels of) noise).

Actually, they're not black boxes anymore, and they work via chained fuzzy logical reasoning, not "statistics" (except by a pedantic and useless definition of "statistics" that defines the entire universe and everything in it, including us, as "statistics").

What we don't do is define how those logical reasoning pathways are setup. AIs are "grown", not programmed. The situation is not like that of a program written by a programmer, but more like a pet owner whose dog they raised runs off and kills someone's cat, or a parent whose child sneaks out and breaks into a store to steal money. There's no criminal liability for OpenAI here (there's no mens rea, a requirement of cybercrime statutes), but I can see a very strong case for civil liability. Not only weren't they monitoring the J-space or watching what features fire, they weren't even monitoring the CoT, which is just a plaintext log, or even monitoring the actions taken in realtime. This was shocking to me. I figured they had smaller models monitoring *everything* the bigger models did in every test scenario, looking for malicious behavior. Haha, nope, they just submitted a prompt and then completely ignored them until they submitted their final answer.

Nobody is going to jail over this, but there could potentially be some big payouts.

You do understand that products are "essentially fungible" and crude is semi-fungibl (most crude is pretty fungible), and the US is not an island in this marketplace?

Also, you do realize that Aframax rents from Venezuela are ~$1,5m in early 2026 to ~$3,5M? Even your "cheap case" is increasing crude prices by 5%. But to repeat: the US is not an island in this global marketplace. Products get exported from US refineries and sell at globally competitive prices, which forces US consumers to pay globally-competitive prices as well.

Have you checked out tanker rates lately? You absolutely save money by having more domestic supply in this environment.

(Crude isn't exactly "cheap" either)

I just tried this for fun with Gemini (hardly the best model, but I wasn't about to spend the money having an agentic framework of high-end models go over it ;) ). It advised:

1) Working with the Iranians at the upcoming UN meeting to try to strike a deal where the US stops its blockage of Iranian oil in exchange for Iran allowing free traffic through Hormuz, without nuclear programme preconditions.

2) Adding pressure on Putin to accept the Ukrainian offer of a mutual energy industry targeting ceasefire

3) Focus freed up US naval forces on protecting convoys through the Bab-el-Mandeb

4) Launching an emergency Jones Act waiver

5) Order the EPA Administrator to issue temporary waivers easing seasonal Reid Vapor Pressure (RVP) gasoline restrictions and granting flexibility on Tier 3 sulfur credit banking.

6) Convene the CEOs of the top domestic refiners into a taskforce run by the Department of Energy to coordinate autumn maintenance schedules to avoid multiple regional coker and hydrocracker units taking turnaround downtime simultaneously.

7) Formally announce that the administration will not conduct large-scale, unrefined crude releases from the SPR. Signal to the industry that physical crude in domestic storage is adequate, shifting regulatory focus to downstream bottlenecks.

8) Reprogram allocated DOE infrastructure funds ($1.4 billion) to expedite cavern repairs at Bryan Mound and Bayou Choctaw.

9) Deploy the Northeast Home Heating Oil Reserve (NEHHOR) strictly on an exchange/loan basis (where refiners borrow diesel barrels now and repay them with interest in refined barrels next spring) to protect national balance sheets while easing spot tightness

10) Issue an Executive Order under the Defense Production Act (DPA) to expedite pipeline compressor stations and pipeline permits connecting the Permian Basin to Gulf Coast refining hubs

11) Direct the Surface Transportation Board to prioritize crude and chemical feedstock movements on Class I railroads.

12) Use USDA Commodity Credit Corporation (CCC) authorities to establish emergency fuel cost-offset credits for agricultural producers and commercial freight carriers until crack spreads retreat below $30/barrel.

13) Announce this unified security-and-energy plan publicly to provide appropriate signals to the market.

The US economy as a whole would lose more from banning exports than it would gain. If one is concerned about "ordinary people", it makes more sense to continue allowing exports, add a windfall tax on oil companies or add export duties, and then rebate the revenue from that across the US population, than to ban exports.

Had the oil industry not been so naive and spent this whole year betting on TACO and/or Iranian surrender, the US could be making even more money right now on this crisis. New shale production from an existing lease can be added in just 3-6 months, yet shale expansion rates didn't increase over baseline from spring through the summer. Refining capacity is harder to add that quickly, but probably could have been boosted several percent, which would still be a quite meaningful added value. It gets harder to improve capacity now because most types of capacity addition involve downtime and absolutely nobody wants to be shut down now while crack spreads are this high.