In many cases, the reason you cant do that, is because of the requirements of the seminal study in the first place.

Things like lifetime cohort studies, for instance, (where are you going to get another 5000 people to track for a lifetime study of a once in a lifetime event? A time machine?) or where very specialized equipment that costs a small fortune to produce (like the stuff at CERN) are at play.

Think about what you are actually saying, and then think more critically about the replication crisis, and then think about the current state of academia more like an experiment that is not performing according to expectations. (specifically, the expectation is that impact factor and impact scoring are sufficient controls to combat and control fraudulent papers proliferating and poisoning the credibility of the entire endeavor.)

Current processes are geared to explicitly maximize new work, even though the actual quality of that work cannot be verified, and is increasingly having problems with actual quality. (with perverse incentives on the rise to actually do the opposite: actively degrade quality. See for instance, the hackjob work done by private interests to undermine "undesired" findings, such as about our climate, and human impact thereon.)

Again, this is because of a fundamental failure to appreciate the value of boring replication work, which is exactly what I suggested.

Boring replication work combats both kinds of problem, but we do not give it the valuation it deserves.

The reason current polices are geared to maximize new work, is due to the resource scarcity with which to do meaningful work in the first place (it's very hard to get the funding to follow 5000 people for 50 years to see how the removal of tetraethyl lead from fuel has changed human behavior, for instance), which is another way of saying that there just isn't enough funding to study the things that need to be studied, let alone verify the findings of the things we can fund to study.

The people holding the purse strings are still politicians, since they set the size of the award pool to start with.

So far, your arguments have been "Refusal to see the forest, for the trees" and "Insisting nothing is wrong, even with alarming evidence to the contrary in your face."

Am I saying that your course of action is incorrect, given your position? No. You are and have been doing what is necessary in the face of resource scarcity, to get as much science done as possible with the best quality you can manage with those resources.

But does it create the replication crisis? Yes. yes it does.

Scientists are humans, and humans are prone to certain modes of mental derailment. There is a very strong bias that the current system is functioning well, even when many outstanding measures indicate it is not. (this study from the summary, and numerous others, for example.)

Why is that, I wonder?

Why do you insist that nothing is wrong, or that dedicated replication teams are so unglamorous, as to be worthless to academia-- or, in your words, "The things you give undergrads" ? (as if it is work "beneath actual scientists" rather than a valuable and indispensable tool in that process)

More pointedly, you assert that things are fine as they are, since "We still catch fraud"-- even though the data suggests that fraud is INCREASING, and catching it is falling behind, which would indicate a failure in methodology...

In fact, recent studies have indicated that its becoming so common, that its become an actual industry, and increase at a rate that very clearly indicates that this is NOT being adequately controlled:

https://www.science.org/conten...

Yet you insist that the methodology is fine-- Why is that?

Again, I would conjecture, it is because there is a startling degree of disdain for "mere replication of findings", combined with an awards system that actively prices that work out of the process, with no system in place that *ADEQUATELY* polices the problem. ("Adequately", because this rate or error is increasing at this very alarming rate) This is abundantly clear from widespread findings in the academic field, like the study I just posted a story about-- its just one of many.

Impact scoring (including impact factor), is very clearly not a sufficient control for this process. If it was, this result would not be appearing.

The scientific process would suggest that this is an observation, and that the next step is formulation of a hypothesis for testing.

I have provided one for you, and it can be tested. Why has this kind of thing not been proposed and examined with the appropriate process?

I can appreciate that there are precious few resources to allocate, but this kind of thing can be tested in small scales for performance quality measures.

It's what's called for by the scientific process, so why has academia resisted it so much?

Or, does academia think its own policies are somehow above the very process they use to wrest truth from bias? (again, scientists *ARE* humans, and humans *DO HAVE BIASES.* Things like "Sunk Costs Fallacy" and pals, spring instantly to mind, given the battle to attain tenure and recognition in a field. "Appeal to authority" also comes to mind, with rhetoric about Impact Factor and Reputation Scoring, in clear contravention of very observable trends.)

Try to be more objective about the degree and severity of this problem, and the outstanding need your vocation has to maintain its rigor and value to mankind.

Especially in the face of a very well funded, concerted effort to undermine that work.

Impact score is literally the number of times a paper is cited by other papers.

Instead of pretending it's magic, instead realize what happens when studies are *not* replicated.

A single study is conducted, and because it is the seminal paper, it gets lots of citations in related works.

Assuming an academic forger is smart, and does not make outlandish claims that break ancillary studies, they can go undetected for decades.

Like the work behind the amyloid hypothesis.

The methodology currently employed grants awards to very skilled fraudsters, in increasing quantity and severity, as suggested by this study, and supported by the observable lack of replication being done.

The politcians I mention provide an insufficient financial resource to provide for the degree of replication needed, replication scientists dont get near the impact scores of seminal paper authors, and conversely, through the the process you laid out, dont get funding approved, leading to them getting even less funding, because you cant realistically do science on a 0$ budget to get the impact scores you need to be awarded that funding. You've created a singularity.

To have competative impact score ratios, there would need to be dedicated 'refutation firms', that predatorially kill published findings, and get citations for doing so. Those firms would need good premises and equipment equally on par with the vanguard, and in many disciplines, that's an equally costly outlay that may require a fed budgetary line item.

We dont have those, and we dont have those for reasons related to the insufficiency of impact score as a proxy for merit, combined with generally insufficient funding overall.

The issue is the expectations.

People expect these things to be thinking entities, providing an independent perspective on whatever you submit to it. A great deal of care must be taken to make it clear and culturally understood that these things are like very very fancy parrots more than an independent human. Which is an uphill battle because we want to anthropomorphize *anything* at the slightest hint, and a puree of training material blended with your prompt and anything stuffed into the prompt (context/RAG) in rather convincing natural language is just really likely to make people think it's more than it is.

The Scrabble analogy is not that great, as anyone can plainly see they are just letters, but to understand the resemblance of LLM to that, you have to go beyond how it *looks* and dig into the nuance of the workings of it, and even then some people have fallen into the trap of "well maybe humanity is nothing more than this anyway".

If applications were automation friendly, then sure.

Problem is the paradigm of application development has been monolithic applications, making it hard to handle workload 'piecewise'.

So the industry has been coming to the realization that Agentic LLM is damn near impossible for these sorts of applications, and have pushed the 'MCP concept', which if you get into it, is roughly like defining CLI interfaces for your application to let a text oriented orchestrator reach into your application to do some work and potentially mix your functionality with other applications. Whatever value the 'agentic' might offer, the ecosystem has to shift to consider that sort of interface must be accommodated, and thus the 'OS' becomes the place to go, as the platform dictates these sorts of design guidelines.

Though I do wonder what MS does as a UI paradigm, they already tend to be pretty bad about UI design, this could be a pretty severe worsening beyond that.

AI doesn't listen though, it regurgitates.

There's as much engagement as writing in a journal no one will ever read. A conversation with yourself is every bit as useful in this context as throwing your text at an LLM.

A conversation seeks another active perspective, an LLM has no perspective, only the ability to dispense a puree of content launching off of whatever prompt that fed it. There are applications for this, but psychotherapy is absolutely not one of those, and substituting an echo chamber for actual human engagement is a recipe for very bad stuff. Online communities are bad enough at unhealthy echo chambers as it is, it's a terrible idea to completely close the loop so a person just hears what they said being said back to them in a different way.

To view LLM as a valuable outlet for mental health is begging for trouble. It doesn't have perspective but it *looks* like it gives perspective. You have only what you brought with you, but you start thinking that someone is agreeing with you no matter what you are saying.

Just like a random dude on the street cannot just say "I can provide psychotherapy services", it absolutely makes sense to also apply those sorts of guardrails to AI, and currently AI is not even vaguely geared toward psychotherapy and just resembles it near enough to be pretty dangerous.

But it only gives the 'illusion' of a response, with an emphasis on reinforcing *whatever* the prompt directs. This can be catastrophic for mental health scenarios, where the provider needs to challenge the patient as appropriate.

Sure, have your chats, but no one should ever call it a substitute for therapy from a provider. Nor should a provider just foist a customer into an AI chat to get more billable hours for irresponsible behavior.

No. The *correct* way to fix this is to resolve the root cause:

How funding is awarded.

Currently, the paradigm is 'publish or perish', because science funding is only handed out to 'rockstars' by politicians who dont understand the fundamental value of boring replication work.

It is the toxic combination of 'I can only do work if I publish first and publish often!', and 'There is nobody checking my work anyway; nobody has the funding to do verification! that leads to this perverse outcome.

Further restriction to 'vip rockstars only!' Is a gross misunderstanding of the root problem, and would be heaping jetfuel on top of the dumpsterfire.

Turns out, you actually need non-rockstars--Lots of them.

And to have them, you have to fund them and their laboratories.

Oh, how awful! You cant have 'only cream'. /s

The sooner this is realized in policy, the better.

It's not just the culture of not giving a fuck, it's the culture of passing the buck.

If the old Demotivator of "none of us are as dumb as all of us" is true, it's dealing with Indian outsourcing organisations. Whenever something goes wrong, and it goes wrong frequently enough that it's a well oiled procedure, you'll never be able to pin the responsibility on anyone. First they'll all start blaming other teams, then big meetings are called where everyone who can be involved is (more people speaking means less gets said), Teflon coated deflections which end up with dragging another person in to deflect and blame someone else until the meeting runs well beyond time which is used by people to leave before any tasks can be given to them, let alone any responsibility. Above all else they avoid any actionable items from being created, insisting that there must be another meeting and that everyone must agree before anything can be done.

Any attempt to use an American or Australian style of "cutting through the bull" is treated as if you've just murdered their nan and bummed their dog. Which inevitably leads to complaints about you being "rude" which is another means of deflecting responsibility.

This, I'm old enough to remember when payphones were commonplace. First they had rubber cords connecting the handsets, as these were cut regularly they started using braided cords, then steel braided cords (like a high end brake line) then finally steel coil sleeves... and scrotes would still go out with bolt cutters on occasion. All that did was ensure that those who wanted to casually damage a payphone needed to put in more of an effort to do so than a pair of scissors. It reduced vandalism but didn't stop it.

When Australia's public telco was privatised in the late 90s, one of the first things they stopped caring about (I mean after customer service, their staff and actual telephony service) was the payphones they were supposed to maintain. In the early 00s they argued that mobile phones had become ubiquitous enough that they didn't need maintain the public telephone infrastructure. As this was the same government who privatised them it was bought without question.

In the context of beautiful women, sure the "by the gods" bit should be "go out and meet actual women".

This will require some activity on their part, like bathing which might be a hurdle.

You've seen X since Musk took over?

It's gone from a barely contained cesspit under Dorsey to a hive of hatred and scum under Musk. If anything he's demonstrated that firing 80% of the staff results in a massive decrease in product quality.

Body shape ==> Race for people in the Late Middle Ages. The kind of people whose grasp of biology is pretty weak.

That's foolish. If you make all models look like you, the US world will be polarized even more than it already is. The major reason why America is so full of hate these days is because social media curate a personal world view for everyone that makes people and ideas outside their cocoons look alien and dangerous. If all you see is body types that look like you pretty soon you'll have all new kinds of body type racism.

Fewer showers? That's ridiculous! Personally, I produce a lot of water after a few beers, and I'm happy to sell that to Google so they can cool their data centers. It's specially processed water, high quality AI coolant. Only the best.