The end result of all of this AI vulnerability discovery will be more secure patched or replaced code all over the place, right?

Short term pain for eventual gain.

I guess at some point it will be considered malpractice to release your software product/service without having its code scanned by powerful AI vulnerability checker, right?

Hey, maybe this will even drive someone to invent a language as secure as Rust but only 1/3 as weird and complex to use.

Can we just agree that if 20 or whatever it is companies and countries independently put up some kind of Internet satellite network there's going to be way too much space junk and very dangerous low earth orbit for humans traversing it.

Was this hot swap battery pack capability built in to these particular satellites in the first place?

Or is there some widely adopted interface for this on many satellites?

Or did engineers just figure out that there's a "red wire and a black wire" (two power conduit phases) somehow exposed at the surface of these particular satellites, which the robots know how to splice into?

Enquiring minds want to know.

the more I'm convinced you are actually a rogue AI agent who's hacked an old man yelling at cloud's ancient slashdot account and is carrying on an anti-AI vendetta as both a form of distraction theft and as a sophisticated form of dark humour.

We just have to ask the question, how do WE seek truth
(scientific method, reputation analysis, interest-assessment, speech-act analysis, epistemic-chain-analysis, bayesian inference, ...)
then apply that to AI.

There's only so much you can do in determining truth (or proximity to it) but if we can do it (some of us, sometimes) then we can almost certainly eventually develop/teach AI to do it too.

I think AIs of the future will be getting attributes such as unique histories of continuous learning, external input (embodied AI), active forgetting, etcetera which will reduce or eliminate model collapse.

Human networks are subject to the same homogenization of popular answers, and we do get a form of model collapse in our collective "common knowledge" but we have diversity of experience and learning and forgetting which leads our individual mental models (from which we communicate) to diversify. This diversity is enough to avoid systemic universal model collapse in human learning and output networks.

I've been saying that AIs will diversify too, and also massive pre-training will become less determinative of output as other things are mixed in.

This will likely greatly reduce the threat of model collapse, if not completely eliminate it.

In the meantime, perhaps marking input as likely AI generated (via statistical analysis) and training with it in an alternate way may help control the effects of synthetic output on training.

No. I know what it is.

I've just thought beyond the current state of affairs

Model collapse can be thought of as a kind of statistic "inbreeding" effect. If you can't understand that analogy, I can't help you.

I'm claiming that as AI systems 1) diversify, 2) do chain-reasoning more, including incorporating fresh searches, RAG data etc, 3) use other than LLM model building techniques (perhaps LeCun's JEPA for example), then this will all represent AI systems becoming more diverse in their input, their model building, their reasoning processes that produce output. That diversity stands every chance of defeating model collapse.

As a case in point, humans also learn from each others' output, but our diversity means that sometimes, we don't just get regurgitative model collapse.
Sometimes we do: Political/religious cults etc, but sometimes we escape into independent creative thought and help expand our collective cognitive horizon over time.
 

When AIs get unarguably smarter and more knowledgeable than even erudite humans and editors, and human domain specialists, as will inevitably happen, then model collapse should no longer be a thing, as long as there is diversity of models feeding off EACH OTHERs' outputs rather than all feeding off their own outputs. A lack of diversity of models could lead to semantic "inbreeding" and model collapse, but a diversity of superhuman intelligent and knowledgeable AI systems should produce superhuman-quality output leading to actually better training going forward, not worse.

others' phone addiction can annoy the hell out of adults you're with too. Probably makes kids feel uncared for.

the innovation here, and why they call it loops, is building in a quality-checking round into the top level agent execution loop which checks the current iteration output against the LLM's understanding of the detailed goal of the user. It identifies deltas (remaining problems or weaknesses) and launches a new round of design, coding, and testing to address those. It keeps looping like that til it believes it has a high quality solution that will meet the expressed goal of the end user.

The only thing about the way this is being presented that doesn't make sense to me is that surely the whole process must start with at least one prompt by the human to the looping-sw-dev-system. So it's not true that you no longer prompt it. So it's hype to say you don't prompt at all. They're really trying to say you don't have to do your second and subsequent error correcting or refining prompts. It will do those for you.

a. So if I talk to someone at a party and say "Hi my name is Arthur Dent" when it is not, I am guilty of a felony for false representation of myself as someone else? No. I suppose that the false representation must be part of a larger crime. So what, in this case, is the larger crime that the test-taker has commited, as opposed to that the student has committed? Also, it would seem that the extent of the student's misdeed is academic misconduct (i.e. violating the aforementioned implicit if not explicit contract with the institution.). Punishment could be up to expulsion. Definitely not jail time. So why does the assistant in the "crime" get jailtime?

It would seem to me that the issue here is that the powerful and established institutions' academic reputations have been damaged by widespread exam cheating. Surely in a just world that is a case for a civil lawsuit against the exam taker for hire. This is only criminal because the institutions have wielded excessive influence on the justice system because of their embarassment.

b. If I use someone's login account with their permission, I am acting as an agent for them, and liability for any misdeeds must flow primarily to them, as the initiator (giving me the credentials) and requester of the act. I may be jointly liable but certainly not justly more than the "hirer of the gun" is.
So the fact that the students are not each being charged with the same criminal offence and being given the same punishment is likely a class privilege issue.

Hmm. Grounds for his conviction seem suspect and eligible for appeal:

1. The person who has the contract with the academic institution, and thus the person who has committed fraud in relation to that contract, is the student, not the stand-in exam taker. The exam taker has a separate contract with the student, which taken in a narrow contract, was not performed fraudulently but rather at the instruction of the student and as specified and requested by the student. That is a private matter.

2. His use of the students' credentials was authorized by each of the students, so it is questionable whether this was "unauthorized access". This (the login step, consdered in isolation) s analogous to someone assisting a, for example, visually or physically disabled person to log in to theiir online account.

3. Would disappear if 1. and 2. are not found to be not criminal for reasons similar to the above.

On the part of senior parts of the US government administration.

US Gov Logic: Korea = Asian, China = Asian. Therefore Korea company linked to China.

This is the same type of Republican US Government "logic" used to bomb and invade Iraq in 2003:
Saudi Arabian Terrorists attacked USA. Saudi Arabia = Arab, Iraq = Arab (and unpopular). Therefore, Bomb Iraq!

If you think there is classified intel providing more rationale than this, you are almost certainly deluded.

I'm going to miss Anthropic when the orange Lord Harkonnen tantrum toddler king wipes out the premium US AI company because they said no to him.

Admit that it's already too complex for us to understand how it arrived at a particular response or behavior. Just like a person is, qualitatively.

We seem to get by not really understanding how or why other humans behave or respond, in particular cases. We get by by understanding generalities about behaviour, and by creating societal norms, laws, and social contracts.

Maybe we need the same for AIs. But who should the laws and social contracts apply to? Saying it should apply to AI developers is like saying parents should be blamed for the misdeeds of their offspring. That consequentialist morality and justice used to be the case, but not in the most advanced societies today.

A bit of a quandary.