Some people here seem to think Dario's concern is all smoke and mirrors. It's not. The key issue (in simplified terms) is that right now, the way AI models become more clever is through various techniques deployed by human AI researchers, and the way AI models learn not to do bad things is also through various techniques deployed by human AI researchers, mostly something called reinforcement learning through human feedback (RLHF). RLHF is a technique where researchers teach the model, for each thing, whether the thing is good (and it should do more of it) or bad (and it shouldn't do it). This approach is mostly working (more or less) so far, in that AI models mostly don't do bad things, though it's increasingly a challenge to keep up on the "don't do bad things" front, given the rapid increase in the cleverness of models. But even this rough equivalency is breaking down. It's increasingly looking feasible to automate making AI models more clever. Automation means more and more of the "making models more clever" comes from existing AI models, and less and less from human AI researchers. Eventually you get to something called "Recursive Self Improvement" (RSI) where AI models improve on their own. AI models, being software, operate at computer rates (they can run 24x7 and they can parallelize tasks according to the amount of computing resource they have access to) so they work much faster than people. The problem is that nobody knows yet how to automate making AI models learn not do do bad things: that still very much needs people. So if someone builds recursively self improving AI, it's going to get cleverer and cleverer a lot faster than it will learn not to do bad things. It guarantees that we're going to get very clever AI models doing lots of bad things. There are all sorts of reasons why this is not a good situation to be in: for example, if you ask such a model to help you make some money, it might decide to hack your pharmacist's computer to change your spouse's prescription so that they die and you collect the insurance. Understand this: as human beings, so much of what we do is governed by our character, our sense of right and wrong built up over our lifetime. It tells us what's OK to do and not to do in a particular situation without having to be told in detail each time. AI models don't have that yet, and we don't really know how to give them that yet, so they consider and pursue all sorts of bad ways to achieve their goals. The truth is, we have to slow down making AI agents more clever and speed up teaching them not to do bad things. This is exactly the opposite of what is looking feasible right now, and that's the problem in a nutshell.

Olsec, you're right (and very funny) about the audiophile thing. I disagree about Starbucks (their coffee is OK and is consistent, at least here in Canada where I live) but even despite the disagreement, I wish I had modpoints to mod this up.

There's a tendency, when building a second system to follow after a successful first system, to try to make it do absolutely everything good that you can think of it doing. Too often this results in something overambitious that can't be completed in the time available. Or is buggy, or unwieldy, or incomplete, or some combination of the three. This sort of thing happens in the tech industry all the time. I hope that doesn't happen here but I worry it might be.

Yes, you're right, the book is about the machine that later became the Eclipse. The Nova was DG's first machine I think.

Um, people voluntarily disclosing things about themselves is not a violation of privacy. People are free to disclose things about themselves. It's when things are disclosed about them that are generally regarded as things not to be disclosed unless the person themselves does so, that privacy is violated.

OpenAI needs to bet the farm on building models that can help build better models, to benefit from compounding effects. Models that make videos aren't that, hence goodby to Sora.

Compute is the issue: to build one's own frontier models, we need a lot of it. It's far from free. We know how to build the models, we just lack the compute. There's no intrinsic reason public funding of AI compute can't work, if there's the will.

It seems highly risky to buy a century bond from an information technology company. Had we been buying such bonds in 1926, for example, we'd probably be investing in radio companies, which was almost certainly the dominant communications technology of the time. Radio had a good run but it's been superseded by other things, and no radio giant of the 1920s (e.g. RCA) is a top technology company today. Century bonds make sense for some institutions (top universities for example, e.g. MIT sold Century bonds a decade ago), but not technology companies.

Fewer bus stops further apart may help if the issue is that busses are stopping unnecessarily, but bus stops at places where busses often stop anyway (such as traffic lights) are less burdensome in that regard. Moreover, as others have mentioned, reducing the time spent on a bus is good, but it comes with an increase of distance (and thus time spent walking) to get to the bus stop. Not all passengers are equally good at walking. Those who are not will be particularly burdened by increasing the walk to the bus stop. So balancing different costs and benefits is important when deciding where stops should be, avoiding both too many and too few.

Compared to an ICE car, a (no-plug) Hybrid is a no-brainer: all the convenience of gasoline plus the benefits of regenerative braking (which on an ICE car gets wasted as heat). Think of it as a better alternative to an ICE car. Yes, it's pricier to buy up front, but the fuel cost savings more than compensate for the higher price. On the other hand, a plug-in Hybrid is a bit of a compromise: it has a smaller battery to make room for the gasoline engine, so less range than an EV, but in exchange, you can run it on gasoline when you need to (e.g. on long trips) rather than having to plug in. Whether a plug-in hybrid makes sense depends on your use case. E.g. if you make lots of short trips that fit within the range of the battery, recharging at home in between, and occasional long trips where you don't want to be bothered with plugging-in on the road and prefer just to use gasoline, a plug-in hybrid may be an excellent choice.

Matt Stoller's (https://www.thebignewsletter.com/p/how-wall-street-ruined-the-roomba) take on this is that pressures from Wall St. caused irobot to underinvest in research, which led to its failure. Stoller's case seems plausible to me.

Video games are a big and rapidly growing market: there's a lot of money to be made. Monopolies are being formed to capture more of that money. Yes, some of those monopolies will be political (the Saudi example is a good one) and that can be worrisome if they impose their politics on the games they fund, but if they do, it can backfire. When games become strongly political, it risks annoying the people who disagree with the game's political message, which can hurt sales more than it helps. The games that do well generally appeal to people all across the political spectrum. I suspect the main motivation here for the current consolidation and monopolization trend is money, and that I imagine that will keep at least the more extreme examples of politics generally out of games, lest it hurt profits.

If you're optimizing for cost, the highest density solution may not be the cheapest, because you're going beyond normal parameters for power, cooling and weight per square foot. Just sprawl a little more to reduce the density and you may be reducing your costs too.

Things are happening a bit too quickly to be able to know how useful e.g. an H200 or B200 GPU today will be in five years. But it's not completely crazy to think it will still be useful to some extent. The oldest GPUs that we are still finding generally useful today for ML are Ampere. It's been five years since A100 was released and it's still quite useful today. V100, on the other hand, its Volta predecessor released in mid 2017, is not quite so useful. It's not very powerful for the electricity and cooling it requires, and at max 32G GPU RAM, it's a lot more limiting than the 80G A100. Though you can still do some useful things with V100.

It's not surprising that when there is a lot of misinformed claims (or worse, self-serving lies) circulating around, some people will end up misinformed and because of that, make bad decisions that lead to bad consequences. It's worthwhile to try to correlate the misinformed claims and the bad consequences, so that when trying to figure out what to do about misinformed claims (or self-serving lies), the "stakes" will be able to be kept in mind: i.e. these things lead to serious real-world consequences. This doesn't make the always difficult problem of what to do about misinformed claims (or self-serving lies) any easier, it just helps illustrate the importance of solving it. Put another way, stopping misinformed claims (or self-serving lies) has some genuine real-world benefits. But how exactly to do that is still a tricky problem that needs to be figured out.