AI creates a statistically likely response.

In what sense? With a context window that has thousands of words, there is no way to do meaningful statistics. I guess you can vapidly claim it's likely because it created it, but the real purpose of these statistical pejoratives is to misleadingly claim that it's just repeating big chunks of text it saw in training.

More accurately, a LLM creates a distribution over words as an algorithmic technique to generate a sequence of text, but that doesn't have much to do with statistics. That's typical in ML. They normalize outputs for various reasons, but it's often a mistake to assume there is any statistical or even probabilistic meaning. At least in this case, the probability has a meaning as the LLM uses it to drive the sequence.

Our wiring is based on the physical world around us, including complex social interactions.

True. I'm sure they are training upcoming robotic systems on simulations of the physical world. We'll see how that works out.

That is way more detailed and complicated than what the LLMs are trained on.

Probably, but humans have to do a lot of non-language tasks, and it's unclear how relevant this is for the tasks we want these AI to perform. A LLM can't ride a bike or play basketball, but having these skills might not help much for computer based tasks. Also a modern, big LLM is trained on around 10 trillion tokens. A human who spends every moment of their life reading can only read around 12 billion words. Not to mention that the LLM processes those tokens many times as it trains.

Plus, we are born with a lot of pre-training or what ever evolution gave us.

Evolution gave us a great system for learning, but there are only around 6 billion bits in the human genome.

Why are you posting as a coward?

The model isn't choosing anything. That is very obviously impossible. They are deterministic systems.

The same deterministic argument is often made about humans, and it can still be argued that humans make choices. This type of argument is more about semantics or philosophy and doesn't really say much about LLMs.

Not just choice, they also lack motivation. When they're not producing a list of next token probabilities, they are completely inert.

In theory, I could do the same to a human. Nothing in physics says suspended animation is impossible.

Also, as a lot of people don't seem to understand this simple fact, these models are static. They do not change as they're being used. The model remains the same no matter how many pretend conversations you have.

But they do change when they are being used. During a conversation, they gain context. This context can even be maintained to create a consistent personality and/or knowledge. Of course, it's limited, but it doesn't have to be static. We just have the power to wipe it.

These things only change when we change them. They are not developing or evolving with use. That is simply impossible.

That's true of current LLM models, but I'm not sure it's relevant to the article. Also, there are ML results about learning based on operational feedback. This will be probably be incorporated into the upcoming robotic products. These new systems will need multiple mental components just like a human. If fact, we will probably learn a lot about human mental processes trying to get these things to do useful things.

The specifics of the market also means there are a lot of very bad incentives for the insurance companies to use pre-existing conditions or trying to deny coverage after the fact, and very strange and weird price/discount structures.

There's also perverse incentives for regulated but private industry to increase costs as long as they all do it. When your profits are locked at a certain percentage of costs, to make more money you need to increase costs.

You want to know what happened to science? What is a woman then, rsilvergun?

This just proves his point. It's a right wing talking point to try and confuse people about definitions. Scientists try to understand how things work and often define terms to help explain those workings. Sure they often reuse existing terms instead of making up new words, but they also give a new precise meaning. If the theory is successful enough, that definition might even get added to the many definitions already in the dictionary.

So of course, from a scientific perspective, defining a woman is not an easy question. Like many common words, it's not precisely defined and depends on context. A scientist would need to stipulate a definition if they want to say anything meaningful about the subject.

"60% of typical community greenhouse gas emissions are from transportation."

Where did this quote come from? It's not in the NPR story. It's also irrelevant to the content. While CO2 is the major contributor to climate change, it's not been shown to responsible for these types of health problems.

I would be nice if the actual article was available. Historically much of this research is sketchy. Sure they can see if an accident victim had THC in their system and then declare that as causal to their death, but it doesn't mean it's true. At a minimum, one needs to compare to the base rate of accidents for those demographics. It might turn out that THC lowers average fatality rate since it causes users to drive more slowly.

I think maybe the big mistake that's been made is imagining that intelligence is just one model.

Minsky's society of mind was published in 1986. Ironically, he's often blamed for slowing down NN research with his previous XOR result. More recently, we have mixture of experts LLM models which can be considered more than one model where a gating mechanism is learned to determine which experts to use.

there's some interesting new work on the left and right hemispheres as a whole representing two entirely different modes of attention -- ways of attending to the world.

Is this the Iain McGilchrist work? The original split brain work is fascinating but often ignored.

Maybe these models are breaking down because they're trying to bring together too many disparate things and they lose structure because there is no one structure which can do them all.

Perhaps. It might also help to have a more sophisticated reward system that better uses feedback to continuously learn. These models probably don't have any notion of a simple fact that can be verified. A human might speculate/bullshit on lots of things, but knows that some things are simple, known facts that can't just be made up. They know that they should look them up, and if they do it enough they will eventually learn the fact.

Stop it. We all know your schtick. You say this absurd stuff to create softballs for the left. Come out of the closet already. Or is there an Obama poster in there?

It's tempting to get a mac studio with 128 GB. It's less than the price of a single 32 GB 5090. I guess the real competitor is the cloud. API token access is pretty cheap, and it's pay as you use. I'd need a serious use to justify $3500. Probably only makes sense if I need the mac for something else.

What do you run those models with? Do you use Ollama or something else?

If I were to tell you that a shoe had 5 billion atoms in it - would you think that is a big shoe or a small shoe? You do not know because you do not know how many atoms are in any shoe.

Small shoe. Maybe a protozoan could wear that shoe. As a rule of thumb, don't forget Avogadro constant which is roughly 6*10^23. That number of water molecules is about 18 grams, so that shoe would be around 1.5*10^-13 grams.

I think some countries have fines that are a percentage of your income.

This is exactly what anyone the least bit conversant in machine learning could have told you.

Yes it is a story that many with the least bit of knowledge have latched on to. Doesn't mean it's correct.

As for the academics and smaller industry players, I'm sure they are not happy about being locked out of the "real" research. It takes millions of dollars in electricity to do full training experiments which is only feasible for a small number of companies. What's left is mostly playing games with prompts which, while often effective, could be done by high-school students.

At this point, it's unclear if they've hit a significant wall. Obviously they are doing a lot of research. Unfortunately much of it is not public. Time will tell. Personally, I'm supportive of the more academic focus on getting a better understanding of how these models learn, but I'm not optimistic. Even the theory for much simpler models is not very useful.