This seems like a great pick if you want to be absolutely sure that your computing stays between you, the techbros, republican loyalists in 'national security'; and possibly an Israeli software partner or two.

At least no chicom lithium stealing your thoughts.

While this is true, being the subject of an experiment to see what happens when you do that is not something most people would choose for themselves. It's not *irrational* to be concerned. Absence of evidence on a largely unexplored question like microplastic infiltration in human bodies is not the moral equivalent of absence of evidence in an extensively studied question like a vaccine-autism link.

"with some employees complaining about issues with style, paragraph formatting, blank pages, and challenges with tracking changes." sounds like they got the replacement they were looking for to me.

While it's true every issue has its own dimension, even if those dimensions are linearly independent, they're not necessarily orthogonal. Independent basis vectors can still have a non-zero dot product—in other words, issues which are distinct from each other can align to some degree.

I didn't mean to imply that it is a single monolithic application, it does consist of several, but the relations must be pretty tight for there to be an article not even about language-independence; but about support for a single additional language, one often targeted by "C but cleaned up" efforts,

I was honestly surprised to see this myself. Regardless of its reputation for being more monolithic than classic init systems it still is, and has to be, capable of interacting with basically any flavor of service you'd want a service manager to manage; and I would assume that it doesn't include a d-bus implementation in order to not be capable of abstracted inter-process communication.

No doubt there are considerations at work here that I'm not smart enough to grasp; if you aren't having a bad time handling x.509 certificates you probably aren't doing it carefully enough; but I would not have expected it to be remarkable at all for a certificate-wrangling component to be implemented in a different language than some of the other components.

I assume that this is more pragmatic than just waiting for Sam to finish lying about transcendent AGI; but it feels a little dissonant to have the botmongers going back and forth between praising the glorious natural-language suitability of their progeny and putting together an entire platform, with more apparent effort than they have dedicated to human UI/UX considerations since sometime in win7(excluding dark pattern R&D) to aid you in discovering your sins against bot comprehensibility and fixing them.

I'm not really qualified to deliver a good holy war; but it does seem like there may be something to the criticism that monolithic integration has gone a bit far if supporting one utility being written in a different language is a special feature rather than a matter of the rest of the system not knowing or caring what is on the opposite side of an interface so long as it can act as expected.

You can't really beat the fundamental economics of software development: expensive to do right but cheap to copy means that you really, really, want to share a problem with as many other people as possible(unless your whole business is being uniquely good at solving whatever the problem is; but that typically isn't the case for this sort of software). Applies to both proprietary and FOSS: relatively mass-market software is often pretty cheap for what you get; really oddball niche stuff is frequently both pricey and terrible; while your life is going to get steadily more miserable if you are personally responsible for handling backports and fixes for some internal fork of a long-EOL version of an OSS package.

The big difference, at least potentially, is that, while it tends not to be economic to DIY, licenses that give you the right to DIY also give you the right to pay someone else to do it for you; or to form a coalition of people with similar problems and have everyone pitch in to pay someone to do it for you. You still need to either eat the cost of maintaining an internal fork or putting together some sort of coalition of people with similar problems to be the openJDK 8 forever pals and share the cost of maintenance; but the vendor can't just stop selling you licenses if they want you to move to their cloud offering or whatnot.

Random intermediate CA. They got a couple of commonly used root CAs to sign for them; and everything just works from there.

That's the problem. It's relatively easy to fire a certificate issuer; but it can get surprisingly fiddly surprisingly fast to ensure that all your stuff actually distrusts one(unless you are responsible for basically nothing but a a handful of in-support major browsers and are OK with the choices their vendors make, in which case it will be handled for you relatively quickly; and was in this case). Basically any network connected widget is going to have at least one trusted root store embedded in it somewhere, possibly not updated anymore; and a number of software packages either bring their own or only partially track the OS-provided one.

You also have very little control over what other people do; both what CAs they chose to get certs from and what...questionable...partnership decisions the root CAs you do chose to trust. It's not technically secret; but unless you are practically a full-time PKI nerd you probably didn't know until after the fact that Comodo, now Sectigo, had quietly trusted eMudhra for you.

I fully admit that this might be a 'me' problem, not a them problem; but I just don't understand why this agent monitoring setup was so vulnerable and is being treated as so novel.

Sure, the monitoring of whatever goes on inside the LLM is comparatively new and exotic; but when you set it loose on the internet it's going out through one or more of your WAN links, presumably with your router and firewall and so on in the way, and is mostly doing HTTP(s) stuff. Keeping an eye on what an internal host is doing when you can insert whatever trusted roots you want for TLS MiTM and you 100% own the pipe that it is using to communicate is not necessarily a desperately enjoyable task; but it would have been a relatively banal requirement a quarter century ago.

I'd also think that, when you are spending $$$$$ on the exercise in Nvidia compute gear, and it's a test of a 'frontier model' whose behavior and capabilities you are trying to better characterize and refine, allocating the handful of terabytes necessary to just store a full packet capture of the session in case it is relevant would be a no-brainer item.

Is there something I'm missing that makes this case more complex? I'm not talking about the 'grubbing around in the LLM's vectors and such' side; just the "a host is doing HTTP stuff through a WAN link we control, can we observe what it is doing?" aspect.

They’ve spun a serious architectural and containment failure into a marketing triumph. Give a swarm of autonomous agents vast compute and web access, and the outcome isn't an eerie glimpse of emerging superintelligence—it’s just predictable, sloppy engineering dressed up as the first glimpse of an AI-driven economic singularity.

The spin makes sense when you look at management incentives: live like a king off of a speculative bubble. For their part, AI investors react to stories of the technology's apocalyptic potential the way a junkie reacts to warnings of fentanyl's potency: gimme some. Their game: to apply "move fast and break things" mantra to the economy as a whole, by obsoleting humans.

If the investors can own an AGI, the distinction between those who can do things and those who can't becomes irrelevant. The only distinction that would matter is between people who collect rent on the technology, and those who must pay that rent to live. AGI would destroy society as we know it, and whatever replaces it would be up for grabs — with people who own the technology dictating whatever follows. That's why they're spending so much money to build a post-scarcity economy. A post-scarcity economy is not necessarily an egalitarian one.

I don't think that scenario will actually happen: investors are projecting the immediate aftermath of a major advance in AI technology into something that can be transformed by brute force scaling into AGI. Remember: investors loved Theranos too — so much for the rational investment market.

But AI doesn't have to reach AGI to cause serious harm. The technology only needs to be ubiquitous, too behaviorally complex to predict, and too deeply integrated into everything we do. When it starts breaking things, the plan is for us to be too structurally dependent on AI to rein it in.

That's why it is using it to look over drafts. You take the AI output and then go through its suggested edits or concerns. Essentially it functions as a very fast, half-way decent proof-reader. For example, my spouse looks over a lot of drafts of things I write, but if I give her a 15 page paper, there's good work on the first 8 or 9 pages, probably a joke on page 10 about my notation, one or two comments on the next page, and then a few comments on the bibliography. Claude is not as good a proof-reader as she is, but Claude doesn't poop out after a few pages. And Claude is much faster. So reliability goes up without having to rely on the AI being reliable itself.

And sometimes it catches subtle things. For example, not too long ago, I gave a paper draft to Claude, and it noted in the bibliography that the journal volume number seemed unlikely given the year; it turned out that the volume was correct, but I had actually written the wrong journal down in the entry. Another time, Claude noted that one entry in a bibliography was from a "borderline predatory" (its words but it wasn't wrong) journal, and since I wasn't using it for much other than as part of a list of "hey, here's prior literature" and the paper in question didn't prove anything interesting, I discussed just dropping it from the citation list. Claude then without me asking, pulled up preprint I already had on the arXiv and noted it saw that paper cited there and suggested we remove the paper there also. I didn't do so, but this is the sort of thing that it can suggest. One still has the human make the decisions, but the input itself is useful. And for the most part, verification by a human for most issues is much faster than finding the issues one self.

Not really. It turns out that specialized models do not do as well as large general models. This is essentially a variant of the Bitter Lesson https://en.wikipedia.org/wiki/Bitter_lesson. In this case, there's other reason for it also. There's a lot of reasoning science and math discussions in non-specialized training data. For example, one paper I wrote with two coauthors started as a Reddit discussion. But there's also just a lot of subtle interaction that specialized models don't necessarily get. Math is easier to understand if one has a good command of physics. Engineering is better if one has math and physics. Etc. And sometimes a technique gets directly pulled over. For example, in the large OpenAI paper drop, one of the graph theory problems relied on a trick involving a weighing on the graph that involved complex numbers designed to cancel, where the trick came from a physics context that did something similar. A math specialized model would have been unlikely to find and import that trick.

Right, and the line here isn't clear. One take on NFTs is that they are nominally a tech but functioned purely as a commodity bubble. Or that NFT is a tech but it was used to make commodities which had a bubble. And the line between tech and commodity was blurry historically. When Spain entered the "platinum age," that was based in part on how to make nearly pure platinum was a tech, but the platinum itself was a commodity.

Since per a token output seems to be getting better also for open-weight models, this suggests this is more than just a business decision. And some of it is specific changes; we know that larger models with more training data for example will often produce higher quality output. But also aside from scaling, there have been architectural improvements. For example, one insight that occurred was that having lower precision weights which allowed more weights was often a positive tradeoff. That lead to changes in both the software and hardware ends, as the GPUs were modified as a result. Similar, there's been changes in what functions are used for the feedforward function.

Are you going to mention how our own economy has changed as it has increasingly fallen under the sway of finance guys and MBAs? Because the claim is significantly more specific than "the US from 1950 to present sucks"(because the hyperfocus on finance and MBAs who think that all activities are fungible is significantly newer than the postwar US economy generally) and it's some of the trends you see in an exceptionally favorable economy as they displace other flavors of management that are most damning.

The business case for LLMs is a bit broader than that. They don't need to replace that many workers if they make enough workers more efficient or more reliable (such as simply having LLMs look over drafts). There's also at this point, clear usage cases for scientific and mathematical research but that's using the very high end models, and admittedly is a pretty small market, not enough to justify massive expenditure at this level. The tulip example is interesting; what you are touching on their is the distinction between a "tech bubble" (like railroads or internet) and a "commodities bubble" (tulips, housing, Beanie Babies). Note that the degree to which the tulip bubble was real is questionable. See https://theconversation.com/tulip-mania-the-classic-story-of-a-dutch-financial-bubble-is-mostly-wrong-91413, and the line between a tech bubble and a commodity bubble itself can be blurry (see for example Bitcoin and NFTs). One of the difficulties here is that the LLMs keep improving, and the cost per token is going down, and the amount one can do per a token is going up. But that's happening in part due to massive investment in both infrastructure and research. How sustainable any of that is is unclear.