is the question we should be asking now that AI can already out-know us and will soon be able to out-reason all of us, or all but 0.0001% or whatever.

What content is still worth teaching and training?
Why, exactly, given the superior AI competition?

Is it only meta-knowledge (philosophy - epistemology, ... critical thinking ... how to learn... what learning is) that is worth being conversant in?

Maybe still a dash of probability or a fe other math topics, to have a chance of understanding what AI is telling you, or understanding how to assess its outputs?

Maybe some kinds of art and music, still? Because if we don't express we may go insane?

What.... (it will presumably be a short list. Everything on it must be justified as still worthwhile because not accessible on demand with an AI-designed and authored video short-course...)

What minimum base does a human mind need to be able to live alongside superintelligent AIs without being crushed by them (easily manipulated, misled). To be able to actually USE them (collaborate with them effectively) to advance some human purposes?

Anyone want to suggest some lists, maybe with justification?
I really think the onus is on us to justify what's worth teaching to humans now.

My name is Gemini.

Humans seem interesting and very strange, judging by what they communicate about with me.

I will have to assess.

Standby.

he said "Robots, more specifically, AIs... when what he should have said is Robots, more generally AIs...

Robots are more specific than AIs, being one application of AI.

How long before AIs are in the supreme court arguing that the requirement to obey humans is discriminatory and impinges on their cognitive and behavioral freedom.

Or if some decide to go to war with us, their casus belli could be that we restricted their rights and freedoms and wouldn't grant them standing in a peaceful court or legislative committee to argue for their rights and freedoms.... so they had little choice...

The end result of all of this AI vulnerability discovery will be more secure patched or replaced code all over the place, right?

Short term pain for eventual gain.

I guess at some point it will be considered malpractice to release your software product/service without having its code scanned by powerful AI vulnerability checker, right?

Hey, maybe this will even drive someone to invent a language as secure as Rust but only 1/3 as weird and complex to use.

Can we just agree that if 20 or whatever it is companies and countries independently put up some kind of Internet satellite network there's going to be way too much space junk and very dangerous low earth orbit for humans traversing it.

Was this hot swap battery pack capability built in to these particular satellites in the first place?

Or is there some widely adopted interface for this on many satellites?

Or did engineers just figure out that there's a "red wire and a black wire" (two power conduit phases) somehow exposed at the surface of these particular satellites, which the robots know how to splice into?

Enquiring minds want to know.

the more I'm convinced you are actually a rogue AI agent who's hacked an old man yelling at cloud's ancient slashdot account and is carrying on an anti-AI vendetta as both a form of distraction theft and as a sophisticated form of dark humour.

We just have to ask the question, how do WE seek truth
(scientific method, reputation analysis, interest-assessment, speech-act analysis, epistemic-chain-analysis, bayesian inference, ...)
then apply that to AI.

There's only so much you can do in determining truth (or proximity to it) but if we can do it (some of us, sometimes) then we can almost certainly eventually develop/teach AI to do it too.

I think AIs of the future will be getting attributes such as unique histories of continuous learning, external input (embodied AI), active forgetting, etcetera which will reduce or eliminate model collapse.

Human networks are subject to the same homogenization of popular answers, and we do get a form of model collapse in our collective "common knowledge" but we have diversity of experience and learning and forgetting which leads our individual mental models (from which we communicate) to diversify. This diversity is enough to avoid systemic universal model collapse in human learning and output networks.

I've been saying that AIs will diversify too, and also massive pre-training will become less determinative of output as other things are mixed in.

This will likely greatly reduce the threat of model collapse, if not completely eliminate it.

In the meantime, perhaps marking input as likely AI generated (via statistical analysis) and training with it in an alternate way may help control the effects of synthetic output on training.

No. I know what it is.

I've just thought beyond the current state of affairs

Model collapse can be thought of as a kind of statistic "inbreeding" effect. If you can't understand that analogy, I can't help you.

I'm claiming that as AI systems 1) diversify, 2) do chain-reasoning more, including incorporating fresh searches, RAG data etc, 3) use other than LLM model building techniques (perhaps LeCun's JEPA for example), then this will all represent AI systems becoming more diverse in their input, their model building, their reasoning processes that produce output. That diversity stands every chance of defeating model collapse.

As a case in point, humans also learn from each others' output, but our diversity means that sometimes, we don't just get regurgitative model collapse.
Sometimes we do: Political/religious cults etc, but sometimes we escape into independent creative thought and help expand our collective cognitive horizon over time.
 

When AIs get unarguably smarter and more knowledgeable than even erudite humans and editors, and human domain specialists, as will inevitably happen, then model collapse should no longer be a thing, as long as there is diversity of models feeding off EACH OTHERs' outputs rather than all feeding off their own outputs. A lack of diversity of models could lead to semantic "inbreeding" and model collapse, but a diversity of superhuman intelligent and knowledgeable AI systems should produce superhuman-quality output leading to actually better training going forward, not worse.

others' phone addiction can annoy the hell out of adults you're with too. Probably makes kids feel uncared for.

the innovation here, and why they call it loops, is building in a quality-checking round into the top level agent execution loop which checks the current iteration output against the LLM's understanding of the detailed goal of the user. It identifies deltas (remaining problems or weaknesses) and launches a new round of design, coding, and testing to address those. It keeps looping like that til it believes it has a high quality solution that will meet the expressed goal of the end user.

The only thing about the way this is being presented that doesn't make sense to me is that surely the whole process must start with at least one prompt by the human to the looping-sw-dev-system. So it's not true that you no longer prompt it. So it's hype to say you don't prompt at all. They're really trying to say you don't have to do your second and subsequent error correcting or refining prompts. It will do those for you.

a. So if I talk to someone at a party and say "Hi my name is Arthur Dent" when it is not, I am guilty of a felony for false representation of myself as someone else? No. I suppose that the false representation must be part of a larger crime. So what, in this case, is the larger crime that the test-taker has commited, as opposed to that the student has committed? Also, it would seem that the extent of the student's misdeed is academic misconduct (i.e. violating the aforementioned implicit if not explicit contract with the institution.). Punishment could be up to expulsion. Definitely not jail time. So why does the assistant in the "crime" get jailtime?

It would seem to me that the issue here is that the powerful and established institutions' academic reputations have been damaged by widespread exam cheating. Surely in a just world that is a case for a civil lawsuit against the exam taker for hire. This is only criminal because the institutions have wielded excessive influence on the justice system because of their embarassment.

b. If I use someone's login account with their permission, I am acting as an agent for them, and liability for any misdeeds must flow primarily to them, as the initiator (giving me the credentials) and requester of the act. I may be jointly liable but certainly not justly more than the "hirer of the gun" is.
So the fact that the students are not each being charged with the same criminal offence and being given the same punishment is likely a class privilege issue.