Surely we wouldn't be the first civilization to fall into oblivion after dominating the known world.

However, I'd say over-reliance on any single technology doesn't seem to be the cause of such downfalls, but the inability to adapt to new ones when the old ones no longer work. This happens when the system is so complex or fragile that the operational regions and organisms don't obey orders from the effective control center, and this center is no longer capable of forcing them into compliance. The society then decomposes into separate independent subparts, as it happened to e.g. the Roman empire.

I see two possible outcomes to computing with AI: either the increased processing power allows a leading center to hyper-control society and further cohere our civilization, or it allows opposing centers to rise and challenge the current lead, and they fight for dominance. The recent geopolitical trend seems to be towards the latter, but the effects of recent developments in AI remain to be seen.

Yes, it's the Luddites all over again. No one remembers that the Ludittes had very good personal reasons to oppose technology, and that they suffered under the technological change.

People with affinity for STEM will be fine; even if AIs are very good at making new discoveries, the most important part of any new field of knowledge is realizing what areas are worth exploring; and AI will need to be lead to those areas for quite a long time. Boffins will continue to thrive in such an environment.

What worries me is what will happen to people who are not able to do science. Hospitality and healthcare may indeed thrive, but not everybody is capable of those; and artistic, creative types will suffer when their jobs can be almost equaled at 100x times the rate of production. What will happen to people without any remarkable skill, when their work can be made cheaper and faster by a skilled worker controlling an AI?

You hit the nail on the head. Throughout human history, every day we have needed to provide for food and shelter. Our brain evolved to make us better at anticipating where we would get them and create ways to secure those needs.

The thing is, we now have a new technology - computing - that allows to delay all kinds of work, and we still don't know what all the implications to human society will be.

Before the printing press we had ways to store language in physical form and store ideas; but efficient printing made the process efficient, accelerating our capacity to do science and improve our knowledge. This led to new physics discoveries that brought us better energy management, creating automated machines for physical work. Before steam engines, there were machines that used energy to automate work, but they were limited to specific locations and tasks, such as mills, ploughs and cranes.

Now we have a technology that automates the application of knowledge to virtually any kind of work, physical or mental. Whether AIs have thoughts of their own or not, the fact remains that software is capable of condensing large areas of human knowledge (including STEM knowledge) and applying it repeatedly without direct human intervention. And as you said, we will not have to do again the work to understand those task in order to get their benefits. This will again accelerate how parts of our society function, which will again adapt around the new processes created around that increased efficiency.

Perhaps -- but you would not face charges for the specific crime of "aiding and abetting".

For the example of leaving your car unlocked, you'd only have legal trouble if you proceeded to file a fraudulent insurance claim, and in many states would not incur any tort liability

Thus explaining why "UK patent law is currently wholly unsuitable for protecting inventions generated autonomously by AI machines.”

[Citation Needed]

The crime of "aiding and abetting", in any sane nation (and in the USA) has a mandatory component of mens rea.

The mandatory reporting in federal law applies to FFLs (registered commercial dealers) only.

Individuals are not held criminally liable for the actions of thief, regardless of what was stolen -- a car, a knife, a gun or a porn stash. With firearms specifically, federal law protects owners in the event of a stolen weapon.

California and the handful of other states with a "gun theft reporting law" generally do not explicitly impose civil liability for the actions of the thief, though California is moving in that direction. There are proposals, like LA's SB216 to make owners liable even if they do report in a timely manner, but these are long shots at best and unlikely to survive a court challenge.

I see in the article mention of "sending a letter implicitly threatening the subscriber with public exposure as a pornography viewer", but unlike Prenda which made a point of going after consumers of the more outré genres, the article doesn't really mention what kind of porn BitTorrent users are being sued for seeding?

Since there's little shame in vanilla porn, my assumption is that if Strike 3 is in it to get settlements, they would be going after less mainstream content, and not bothering with people who download stuff akin to "Devil In Miss Jones" (the 2005 remake, not the public domain original)

Don't worry for them, phone tower companies have it covered.

Clever criminals leave their phone home anyway.

I can ask any old search engine to give me a recipe for chocolate chip cookies. I can't ask a search engine to then double the ingredients after it provides me with the recipe. That's not merely "combine the multiple found snippets".

We have a different definition of learning from the training set. The LLM is able to double numbers because it was trained on examples of doubling. If you trained it only with those examples and it learned that x2 means doubling, it would not be able to infer that x3 is "adding three times" and x4 is "adding 4 times" unless you also included examples of tripling and quadrupling. The 'learning' it can exhibit is limited to patterns found in the corpus of input data.

LLMs don't have the capability to deduce new facts from the facts it knows, like a symbolic search engine would; only to probabilistically generate strings or images that simulate such generation. The process is completely different. And it's limited to probabilities over facts in the training data; the prompt is used only to calculate posteriors on those probabilities, not to generate new ones.

I've seen such a capability demonstrated with my own eyes. I pasted a portion of the manual for a proprietary DSL that isn't on the Internet into my context and asked deepseek-coder-33b-instruct to write a program in a language it knew nothing about.

However, as soon as you clear the context, the model will again return to knowing nothing about the language. That's an instance of the generating content most likely to match the multiple levels of activated patterns I talked about. It can create results more elaborate than a search engine (which merely returns the original document unchanged), but it's based on the same principle. It's using the content you provided as input, but not learning it. It won't remember a thing about your DSL or the written program, much less generalize about them for other new prompts.
It has not learned in any meaningful sense of the word, which necessarily implies the information becoming part of the permanent knowledge of the model.
Even if you never deleted and kept the DSL spec and the conversation forever in the context, it would be akin to memorising it rather than learning it, a weaker form. You need something like a LoRA to have the model acquire new knowledge, as only then it becomes part of its training data and can be used as new knowledge.

This perspective is a fundamental misunderstanding of what LLMs are.
On the contrary, it's the result of careful consideration of how LLMs operate and reflection on the observed results.

The point of the technology is generalization, the ability to apply learned concepts. It isn't about cutting and pasting snippets of text from a dataset.

I didn't say that it's merely cutting and pasting snippets. As I mentioned, the model has the capability to use learned language to combine the multiple found snippets into a single coherent discourse. But their discourse *is* essentially a regurgitation of the many items of content retrieved from the prompt; some as text snippets, and others as more complex patterns learned directly from their training corpus.

But if you think that the model creates knowledge beyond what's provided in the training data, you're the one with a fundamental misunderstanding. What you call "generalization" is a codified compression of the trained corpus; that compression happens to capture patterns in the input documents at multiple levels - some at a surface syntax level, others connected to more abstract concepts that humans used to create and classify the content (such as style, emotion, and the meaning of the topics themselves).

When you apply those compressed patterns to new content, such as an input prompt, it activates the most relevant of those patterns, and generates the content most likely to match the multiple levels of activated patterns in the context of the current generation point. But the models in their current form have no capability at all to create new patterns at runtime based on their applied use, i.e. no memory and no method to reason about what they see.

So you're be mistaken if you think they have any capability to learn from content that was not part of their training data; they need to be retrained with more unit data in order to acquire such new content. Maybe in the near future there'll be a way to have models with actual online learning that get new knowledge directly from their own interactions, like they do now offline with RLHF.

The best way to understand what what LLMs are doing is treat them as a search engine that actually works as intended, retrieving multiple results from its corpus of training documents. Thanks to modern languages processing techniques, they are capable of combining several results in a single narratively coherent reply. Just like a search engine though, the quality of the talents is limited by the quality of the documents provided.

LLMs now have the advantage of not being tainted by SEO techniques and in-place advertising (for now), so we get to experience something similar to how Google worked when it was first released: an unbiased index to all the knowledge humans have shared on digital media.

For the second time we'll have a small window of opportunity to see what it's like to have all that knowledge available, until it's again made unusable by the same market forces that poison the pool of content for small personal gains. No need to blame regulation there, which could actually have an opportunity to reduce that degradation if done well.

Are you joking? The *whole point* of Asimov's Three Laws of Robotics was to demonstrate, through his
robot stories, that a small set of simple rules could never work too control artificial intelligence in a complex world full of ambiguities.

First, new AI-friendly programming languages will be created, that are simpler for LLMs to learn. Once developers have assistance of the model to create and understand code, easy-to-read concise PLs won't be that essential; but unambiguous precision will be. Code snippets will become more write-only.

Second, business-level programming will become even more dependent on composing pre-built, well-tested library components. Programs will become less of a logically coherent cathedral with solid pillars that (tries to) solve all the needs of an organisation, and more of a camp of communicating, loosely connected tools that each serve one single concern and actually solves the needs of a worker or small team.

Third, thanks to the previous two, most programs won't be built and run with the current enterprise lifecycle of writing code to specs, debug in a development environment, then release; it will become integrated with the AI platform instead, running in the background without the code ever abandoning the environment where it was written. Multimodal AIs like Gemini are capable of writing ad-hoc user interfaces, built on the spot to explore and consume the data according to some current user business needs. Many of the tools will be transient, single use widgets that solve a simple task at hand following the specifications provided by the end user, just like information workers now are able to create spreadsheets to define ad-hoc processes to keep and model data to solve their work. In this scenario, exposing the code is not something that will be needed usually; only to the point that it's needed to check that operations on the data are using the right logic.

Will traditional programming disappear? Of course not; just like system programming is still needed, someone will have to write the well-tested components that the AI combines; and someone will need to keep in check the architecture of more complex combinations of tools that are used in a stable way through time. But for most small tasks, users will finally able to create their own tools for most of their simple and moderately complex information processing needs.