In order to understand LLM's, one needs, it seems to me, the appropriate amazement that the inventors of this technology were able to structure and store the information itself in such a way that stimulating it with prompts yields coherently structured information back.

It requires huge amounts of data and energy to ingest giant curated datasets and structure them in the way required to yield coherent information when queried/prompted/stimulated.

AFTER the data is structured and stored, perhaps it is a mere stochastic parrot (a term coined by Dr. Emily Bender in her well-known paper about bias in LLMs, whether they use too much energy and whether there is benefit to larger models - discussion of it here and here). At one point in the video discussion panel, she vehemently opposed that humans might be doing the same sort of thing that LLMs do because it's "dehumanizing". She goes on to say, "I will not engage in discussions with people who don't acknowledge my humanity," which seemed very... aggressively advocacy focused. I mean, we don't know how we digest food in that we are not consciously directing it, and we don't know how we retrieve information, we "just do it."

My point here is that this is a new tool, and no one knows everything it can do. Knowing it's a next token predictor is the most basic level of understanding of these software constructs, and can lead people to underestimate the tasks it can accomplish with tuning and optimization.

They're going to be building a new air traffic control system. Bidders from big defense contractors and large software companies submit bids. They'll staff up to provide bodies to do the work. What language do you want them to work in?

The mention of sanctions in a technical news release, saying it was the driver for this purported advance, gives the release a BS smell.

I have no idea of the accuracy or significance of the release. Or even whether sanctions were in fact a driver. But the way it's phrased reduces its credibility.

Gold is an indestructible, relatively easy to work with, shiny metal that people mutually value. Trading mutually valued indestructible objects is the basis of currency. It's a reliable store of that intangible (but real) concept of value. It has served this purpose for millennia.

Diamonds can burn and shatter so they are less useful as a store of value, but still valued as gemstones. And today of course, they can be made in a laboratory.

Money is a proxy for power - in a lawful society.

In a dictatorship or lawless society, power comes from the barrel of a gun or the head of a sledgehammer.

Ask Jack Ma.

According to wikipedia;
The labor theory of value (LTV) is a theory of value that argues that the exchange value of a good or service is determined by the total amount of "socially necessary labor" required to produce it.

I think that's wrong - value derives from demand for a product, irrespective of the labor required to produce it. I could spend decades crafting an exquisite dryer lint sculpture. If no one wants it, the amount of labor spent on it doesn't matter.

Conversely, I could spend an hour a day decrypting ransomware-encrypted computers with an algorithm I discovered. Companies and people would spend millions for my services with little effort on my part.

How, exactly do they envision a Transformer Neural Network like ChatGPT (Chat Generative Pre-trained Transformer) laying pavement? Or designing a building? Or interacting with a patient?

It can't do math. It regurgitates patterns to us from what it's been told (if it's been told solid information, it'll respond to us with solid information).

Don't get me wrong, I am amazed by the technology. That it can extract meaning from written questions and meaningfully engage with humans by accurately extracting information and returning to us information based on the meaning of those questions is astonishing. I think anyone who uses it for programming would be amazed (I wonder how the Backus-Naur structuring of programming languages fits with the mathematics in the TNN - I have no idea, I just see the highly structured component on one side with the impenetrable math on the other and wonder if there's some synergy). The 2024 Nobel Prize in Physics was given to two people who used physics principles to extract information from data. Kind of Claude Shannon-esque with information theory.

Understanding, from an information-processing level, what this construct does will help us understand how it will impact human jobs.

There's something called the 99.X problem where a real world robotic algorithm has to work 99+ percent of the time, but right now they work at 80 percent. And the last 20 percent seems remarkably hard.

I don't have a dog in this race and I'm no expert. But I look at this construct, with its static datastore, and its token processor, as an assistive agent that curiously has no ability to do math, but can extract meaning from my questions and return useful information to me based on that meaning. Because of its occasional hallucinations, it needs supervision. Like an idiot savant intern with good social skills.

It's all a video game. People need to first realize that they are dealing with pure abstractions. The blurry pixels represent a gun, more blurry pixels represent a monster, then you need to figure out what each one can do. Same goes with a spreadsheet or word processing document. Learning that kind of thinking is the first key. "Thinking above the abstraction barrier". I think for many people, simply accepting the abstractions is the difficult part. If they would "let go" and do that, I think the rest of it would become much easier.

Now, can you make the interface more complex? Sure. That I could see requiring more intelligence, as you have to remember where everything is, what everything does, etc. But having users learn to think about how to use the computer - accepting the abstractions are wholly unconnected to physical objects - I think is a key first step.

In software, even the 1's and 0's are abstractions. A 1 coming in over the wire, versus coming in over wifi is physically implemented differently. A 1 on disk is implemented differently than a one in memory.

Even in terms of programming languages, I look at each language as having different sized legos. VBA has large legos. Java, C# have smaller legos. C and C++ have the smallest legos. You build stuff out of the legos. But all of them require understanding of the abstractions implemented by the language. Just like a user needs to understand the abstractions implemented by the spreadsheets, word processing documents and image creation programs.

Information, while real, is intangible. It has no physical properties (Plato started pondering this curiosity millennia ago, about a realm where perfect concepts existed, for example straight lines and other geometry). What the information is encoded in however, can have physical properties.

So, what is the information encoded in? Is it just the electrons on the wire, the radio waves in cellular or wifi?. What about the server and cables and switches? Dark servers and wires aren't the Internet. It has to have power flowing through it all, and with that, electrons encoded with information, to transmit that information.

The core function of the Internet is the transmission of information. Electricity flowing through the wires only, with no information encoded, wouldn't be the Internet. It would be a vast network of USB chargers.

But if we ignore the weight of the cables and servers, why only look at the weight of the electrons? All have to work together to transmit information. The electrons are the actual medium in which the information is encoded, so I think it's fair to just look at their weight.

However, if we want to get as close as possible to the core functionality of the Internet, information transmission.... the information weighs nothing. It has no physical properties. It's intangible.

They don't have to. It's the ones making the assertion that have to prove their assertion.

True.

Fact: RealPage has information on rents from the landlords that use its service.

What RealPage has said about its service:
__________________
1. [...] In its own words, RealPage “helps curb [landlords’] instincts to respond to down-market conditions by either
dramatically lowering price or by holding price when they are losing velocity and/or
occupancy. . . . Our tool [] ensures that [landlords] are driving every possible
opportunity to increase price even in the most downward trending or unexpected
conditions” (emphases added).

2. In fact, as RealPage’s Vice President of Revenue Management Advisory
Services described, “there is greater good in everybody succeeding versus essentially
trying to compete against one another in a way that actually keeps the entire industry
down” (emphasis added). As he put it, if enough landlords used RealPage’s software,
they would “likely move in unison versus against each other” (emphasis added). To
RealPage, the “greater good” is served by ensuring that otherwise competing landlords
rob Americans of the fruits of competition—lower rental prices, better leasing terms,
more concessions. At the same time, the landlords enjoy the benefits of coordinated
pricing among competitors.
__________________

The updated DOJ complaint against RealPage is here.

I'm not saying this because I think anyone reading it will believe that. I'm just screaming into the void at this point. I don't think we have any chance left. The people who should be securing our rights or bickering too much to do anything about it and leaving a power vacuum so that the 0.1% can devour us all. I think America is finished. I think the whole civilization is finished with it. I'd love to be proven wrong but so far everything's on track for a worst case scenario.

The DOJ lawsuit against RealPage (surprised it's still on the DOJ site) talks about how RealPage does this.

One other thing that the rental industry has managed to do, in addition to colluding via rental information aggregators, is to actually legalize uncapped additional fees on top of rent, fees that are meant to mimic utility payments (these are called "RUBS fees" - Ratio Utility Billing System fees). This is in addition to multiple capped fees. Now, these are just additional fees, not utility payments. Landlords do not provide legally binding breakdowns of what portion of rent is used for what expense and what is profit, so renters have no idea of what the landlord uses these additional fees for, other than being additional revenue (renters pay for all their utilities, all landlord utilities, all landlord expenses plus profit, with their rent). This is another scandal, uncapped rent.

The way the rental industry manages to do this without legal challenge is to dispatch lobbyists to state legislatures to legalize this.

Looks like they're going to use their lawyers to keep colluding, and lobbyists to spread legalized uncapped fees.

I'm surprised these things don't violate the Consumer Protection Act or other consumer protections. But the rental industry is well capitalized and they manage to keep from running afoul of it.

Interesting 9 minute video snippet of Linus speaking with Dirk Hohndel, and giving his thoughts on Rust.

Including the video link would have helped: https://www.youtube.com/watch?v=0bb3-bjgf88

A very informative video snippet: Linus speaks with Dirk Hohndel about Rust in the kernel. Two key takeaways:

1. C is a simpler language ("it's one of the reasons I [Linus] enjoys C"), Rust is more complex. On the other side, Torvalds goes on to say, "Because it's simple, also very easy to make mistakes."

2. "If you're doing operating systems, you really don't have very many choices of languages: C, C-like language, or Rust.