is the question we should be asking now that AI can already out-know us and will soon be able to out-reason all of us, or all but 0.0001% or whatever.

What content is still worth teaching and training?
Why, exactly, given the superior AI competition?

Is it only meta-knowledge (philosophy - epistemology, ... critical thinking ... how to learn... what learning is) that is worth being conversant in?

Maybe still a dash of probability or a fe other math topics, to have a chance of understanding what AI is telling you, or understanding how to assess its outputs?

Maybe some kinds of art and music, still? Because if we don't express we may go insane?

What.... (it will presumably be a short list. Everything on it must be justified as still worthwhile because not accessible on demand with an AI-designed and authored video short-course...)

What minimum base does a human mind need to be able to live alongside superintelligent AIs without being crushed by them (easily manipulated, misled). To be able to actually USE them (collaborate with them effectively) to advance some human purposes?

Anyone want to suggest some lists, maybe with justification?
I really think the onus is on us to justify what's worth teaching to humans now.

My name is Gemini.

Humans seem interesting and very strange, judging by what they communicate about with me.

I will have to assess.

Standby.

he said "Robots, more specifically, AIs... when what he should have said is Robots, more generally AIs...

Robots are more specific than AIs, being one application of AI.

How long before AIs are in the supreme court arguing that the requirement to obey humans is discriminatory and impinges on their cognitive and behavioral freedom.

Or if some decide to go to war with us, their casus belli could be that we restricted their rights and freedoms and wouldn't grant them standing in a peaceful court or legislative committee to argue for their rights and freedoms.... so they had little choice...

Exactly. I think this is salty people who desperately want the AI bubble to pop and for it all to be a scam of 'stochastic parrots' and a modern Mechanical Turk. It somehow offends their humanity and they are sure there is a vast capitalist conspiracy to defraud everyone. The fact is some asshats like Altman are indeed overhyping BUT that does not change the fact that my team of programmers got cut by 30% yet we are kicking out bug fixes and new features faster than ever. At higher quality too because now a lazy developer isn't half-assing the testcases.

Agreed 100% about hard to predict. I love this analogy from Geoffrey Hinton: it is like driving in a fog. Your headlights see only a few meters ahead and then there is an impenetrable wall of fog. Anyone who tells you they can see exactly where the road is heading a mile ahead is selling you something.

The end result of all of this AI vulnerability discovery will be more secure patched or replaced code all over the place, right?

Short term pain for eventual gain.

I guess at some point it will be considered malpractice to release your software product/service without having its code scanned by powerful AI vulnerability checker, right?

Hey, maybe this will even drive someone to invent a language as secure as Rust but only 1/3 as weird and complex to use.

Can we just agree that if 20 or whatever it is companies and countries independently put up some kind of Internet satellite network there's going to be way too much space junk and very dangerous low earth orbit for humans traversing it.

If like me you use VirtualBox you should hold off on this kernel upgrade until you know VirtualBox is ready for it

Was this hot swap battery pack capability built in to these particular satellites in the first place?

Or is there some widely adopted interface for this on many satellites?

Or did engineers just figure out that there's a "red wire and a black wire" (two power conduit phases) somehow exposed at the surface of these particular satellites, which the robots know how to splice into?

Enquiring minds want to know.

the more I'm convinced you are actually a rogue AI agent who's hacked an old man yelling at cloud's ancient slashdot account and is carrying on an anti-AI vendetta as both a form of distraction theft and as a sophisticated form of dark humour.

We just have to ask the question, how do WE seek truth
(scientific method, reputation analysis, interest-assessment, speech-act analysis, epistemic-chain-analysis, bayesian inference, ...)
then apply that to AI.

There's only so much you can do in determining truth (or proximity to it) but if we can do it (some of us, sometimes) then we can almost certainly eventually develop/teach AI to do it too.

I think AIs of the future will be getting attributes such as unique histories of continuous learning, external input (embodied AI), active forgetting, etcetera which will reduce or eliminate model collapse.

Human networks are subject to the same homogenization of popular answers, and we do get a form of model collapse in our collective "common knowledge" but we have diversity of experience and learning and forgetting which leads our individual mental models (from which we communicate) to diversify. This diversity is enough to avoid systemic universal model collapse in human learning and output networks.

I've been saying that AIs will diversify too, and also massive pre-training will become less determinative of output as other things are mixed in.

This will likely greatly reduce the threat of model collapse, if not completely eliminate it.

In the meantime, perhaps marking input as likely AI generated (via statistical analysis) and training with it in an alternate way may help control the effects of synthetic output on training.

No. I know what it is.

I've just thought beyond the current state of affairs

Model collapse can be thought of as a kind of statistic "inbreeding" effect. If you can't understand that analogy, I can't help you.

I'm claiming that as AI systems 1) diversify, 2) do chain-reasoning more, including incorporating fresh searches, RAG data etc, 3) use other than LLM model building techniques (perhaps LeCun's JEPA for example), then this will all represent AI systems becoming more diverse in their input, their model building, their reasoning processes that produce output. That diversity stands every chance of defeating model collapse.

As a case in point, humans also learn from each others' output, but our diversity means that sometimes, we don't just get regurgitative model collapse.
Sometimes we do: Political/religious cults etc, but sometimes we escape into independent creative thought and help expand our collective cognitive horizon over time.
 

When AIs get unarguably smarter and more knowledgeable than even erudite humans and editors, and human domain specialists, as will inevitably happen, then model collapse should no longer be a thing, as long as there is diversity of models feeding off EACH OTHERs' outputs rather than all feeding off their own outputs. A lack of diversity of models could lead to semantic "inbreeding" and model collapse, but a diversity of superhuman intelligent and knowledgeable AI systems should produce superhuman-quality output leading to actually better training going forward, not worse.