An AI suggesting which libraries to use is a major attack vector these days.

And LLM as no idea what a library is or which one to choose, it only selects the word that looks plausible. That will generate some library names that "should exist" but where the actual one is called something else because of whatever (history, pet cat, etc).

In fact, it's so common it even has a name now: slopsquatting

References:
https://www.reddit.com/r/singu...
https://www.kaspersky.com/blog...
https://www.traxtech.com/blog/...

Replace ChatGPT or AI with "autocomplete" and all these AI headlines explains themselves.

Autocomplete loses in Chess!
Autocomplete makes up references!
Autocomplete said something stupid!

>Except that that's exactly what the latest "reasoning" models do.

No, not at all. Reasoning models just generate each word as regular LLM models, then feed the answer into the same (or another mode)l to rewrite the response. One word at a time. There is no AI that thinks or reasons here, it's only really clever way of using the magic of "the next probable word". The only magic of LLM is "choose the next word". There is no thinking or reasoning at all, regardless of what they call the model.

> It "understands" about as much as a typical undergrad.

No, absolutely not. It doesn't understand anything. The "hallucination" that people refer to regarding LLMs is really just an occurrence of the wrong word being selected and as a consequence, it will derail the entire sentence (and even the rest of the reply). An undergraduate would not do that, it would not select a wrong word and then completely miss the mark because the selected word is more associated with flowers than electrical engineering. LLMs will do that without blinking an eye, and even sound confident in their reply.

Exactly. They way LLMs work makes it look like it can think and reason (and it doesn't help that they call the new models "reasoning"-models).
Basically, it can only make a list of plausible words that would be next in a sentence based on all the input. That's it. That's really the magic part in an LLM.

It's a stateless model, giving the same output with the same input.
It doesn't learn.
It doesn't remember.
It doesn't think.
It doesn't reason.
It doesn't understand.
It only generates a list of plausible next words. That's it.

But that doesn't prevent media to publish "LLM did something stupid" or sales people to try to convince customers that it can do EVERYTHING.

It's "obvious" for people who doesn't understand how LLMs work
Most people do not understand what is at the heart of an LLM, and as such, doesn't understand its limitations.

I'll simplify a bit for the sake of keeping this post short, but in essence, the "magic" part of LLM is a model that; when given input as tokens (which are words or parts of words) will return the probability for the next word.
The model itself is stateless, it will always return the same words with the same probability given the same input.
It doesn't learn.
It doesn't remember.
It doesn't think.
It doesn't reason.
It doesn't understand.
It only create a list of the next word with attached probability of each.

Of course the "wrapper" around this Magic feeds all previous conversation with the addition of the new word, to generate the next word, and keep on doing that until an answer is complete. That is why it's still power hungry and time-consuming. Having the option to adjust temperature (not just picking the most probable word) and randomness/seed (don't generate the same list of words).
Right now, with the "reasoning" models, we are going down the same path as we did with XML back in the day. "If you can't solve it using XML, then you are not using enough XML". Basically feeding the complete result of the LLM back into itself (or a different one) to have one more pass to see if the training data can match some outlier or unusual text.
To train these models it takes the amount of power of a small city. The technology is almost at its peak. It will improve of course, but not significantly. If something else than LLM comes along, then we can revisit the topic.

Just to be clear: I'm using Copilot (github) both in a professional capacity and for my hobby coding projects. I also have a paid sub to ChatGPT helping me looking into things I want to learn more about. It's a fantastic technology, and it can really help. But it doesn't really do all the things that people think it will do.

I'm amazed the amount of tech people that have no clue what an LLM is and how it works. I get why the CEOs of the companies and sales people are just running around with PR bullshit, but tech people? Why?
Simple facts like LLMs not having state. You feed the same input and it will always generate the same output. You can add randomness to the input of course, but the model is not changing while being used. It's static. Stateless. Having "memory" is just a feature of the client sending in previous dialog (or a summary of it) together with latest input.

Several times more productive?
That would be true if you are creating Hello World apps for a living.

LLMs are impressive, but it's nowhere near being able to code reasonable sized software projects. It can certainly help with scaffolding and plumbing, but it just can't be trusted to do bigger tasks. A developers job is not to churn out code 8 hours a day. A developers job is to transform human language from the customer/employee into features/abilities in the software, and do that in a way that is structured, extendible, performant and maintainable.

As the parent poster pointed out "...this is _old_ tech scaled up. All easy to find tricks are already in there."

That is what most people today don't understand. LLMs are insanely scaled technology to pretty much reach the peak of what it can do. There will of course be improvements, but the giant leap to add "reasoning" or even something like "common sense" is not going to happen with this technology. Even if they keep naming the new models "reasoning"-models.

Don't forget that Apple Silicon M series CPUs are RISC. They made quite the impact on launch.

Later this year we'll get Windows running on RISC with the Snapdragon CPU (including the NPU for "AI"). Windows tried that a few times before, but this time the times really have changed. Maybe it will work.

RISC is certainly having a comeback :)

Yeah, we'll tell you what you can watch and you'll be happy about it! Be a good consumer and don't you dare own anything...

But to put forward some arguments on the matter. Maybe some people like to own the movies that they like to watch? In case they want to watch them again some time or show them to friends or family? It also comes with a nice bonus that they can alter the movie afterwards to be acceptable for a new country, audience or political climate.
And for those who like to collect stuff, why not movies?

Don't be so dismissive just because you don't understand the reasoning behind owing something instead of just renting it.

I've been hearing about quantum computers and algorithms for decades now, and yet there isn't a single practical example of an asymmetric encryption being broken unless you move down to "any laptop on the planet could break this" number of bits.

All the clickbait articles based on papers that turns out to be false or at least not possible to replicate, have made me take on a highly cynical attitude towards the entire field.

Sales have of course taken this into any marketing regarding encryption and are now consistently referring to AES as "Quantum Safe Encryption", which is just hilarious. It's not like you can just replace asymmetric with symmetric encryption to be "futrure proof", they kinda have different use cases.

There is no way large language models (ChatGPT type) will replace developers. It's a fundamental misunderstanding of how LLM works. Specific products like Github/Microsoft's Copilot can help productivity and it's an interesting technology, but at the end of the day it's just a tool.
Development is taking an ambiguous human language and converting it into specific instructions for a machine. Making hundreds of decisions along the way guided by an understanding of what the customer/employer want to achieve with the solution. Experienced developers will also factor in expandability, maintainability, performance, security and much more. LLM's can't really do that. It doesn't think. At all.

Right now everyone is running around and yelling "We need to use AI". Same as back in the day with Blockchain. Both LLM and Blockchain are really cool technologies, but most people have no clue on details. Most importantly they do not understand limitations and other cons. I spent time on learning about blockchain back in the day and I was pitched quite a few ideas. All of them absolutely pointless to implement with blockchain. It's a bit the same now with LLM. It's a cool technology, but the need for "Human like answers that can be wildly incorrect while appearing self confident" is not as big as one would think. Then factor in the incredible expensive compute both to train and query, and it's not the silver bullet everyone thought it was. Add to that the fact that is does not even attempt to actually "understand" or "think" about the topic, even if it's really excellent at giving the impression that it does.

We are going to get a lot of AI features in the years ahead. Most will be completely moronic, but a few will be useful. However, it will absolutely not replace developers or any job that requires any sort advanced thinking.

It won't even replace jobs that require common sense, because it has none.

Evernote was great, but they kept making bad decisions. I suspect MBAs and Excel-based decisions was a big part of their decline.
I was a paying customer for many years, but switched out of frustration.
For example forcing the Todo-list on everybody. You basically couldn't click "New note" any more in the desktop client, you had to click "New" then click "Note" (and not Task). They also had their constant nagging of upgrading Pro, even on an already paid subscriptions. I had absolutely no use for the Pro features, so it becomes anoying to use their app.
It's pretty clear that they lost their way in trying to creata a good product for customers, ending up trying to force their opinion on how to use the ever increasing featurebloat of an application onto their customers.

It was a good product for a while there.

I use Zeek at home with the Security Onion distribution. It's an Intrusion Detection System distro.

Highly recommend trying it if you want to learn more or monitor your home network.
Zeek is not Snort, or anything like a signature based detection system. It parses the network traffic and writes different type of traffic into easily consumable text files (either text or json). In the Security Onion distro Zeek data is imported into Elasticsearch and accessed using Kibana. HTTP, SSH, DNS, DHCP, FTP, SNMP, SMTP, X509 certs being used, etc, etc, etc You can search for specifics or aggregate information in dashboards. Which device is doing what in what time period, or something like the most noisy device, or even figure out devices with scheduled connections (ex. IoT phone home devices). Security Onion can also store pcap files for the actual data traffic.

It's an awesome way to get to know your local network and to look into details on what is going on. Do you know what your Sonos speaker is doing on your network? What about your internet connected Coffeemaker?

As for Microsoft bringing Zeek into Defender, I'm not sure. They don't have anything even close to competing with Zeek and Microsoft seems to actually commit to open source these days. Really hope they don't ruin Zeek. It's awesome.

Do we know if processing wood into final products and shipping them will require more energy and release more CO2 than its plastic alternative?
I haven't seen any numbers, but I think it's an interesting perspective.

Limiting plastic is obviously good for the environment, but is it automatically better when it comes to reducing global warming also?

Games like this doesn't require a "weak minded drone". Most "free" games are designed to be boring and tedious unless you pay up. They have basically removed the entire purpose of playing games, unless you pay.
That doesn't stop them from getting people addicted first, though.