Nvidia Releases a Toolkit To Make Text-Generating AI 'Safer' (techcrunch.com) 53
An anonymous reader quotes a report from TechCrunch: In pursuit of "safer" text-generating models, Nvidia today released NeMo Guardrails, an open source toolkit aimed at making AI-powered apps more "accurate, appropriate, on topic and secure." Jonathan Cohen, the VP of applied research at Nvidia, says the company has been working on Guardrails' underlying system for "many years" but just about a year ago realized it was a good fit for models along the lines of GPT-4 and ChatGPT. "We've been developing toward this release of NeMo Guardrails ever since," Cohen told TechCrunch via email. "AI model safety tools are critical to deploying models for enterprise use cases."
Guardrails includes code, examples and documentation to "add safety" to AI apps that generate text as well as speech. Nvidia claims that the toolkit is designed to work with most generative language models, allowing developers to create rules using a few lines of code. Specifically, Guardrails can be used to prevent -- or at least attempt to prevent -- models from veering off topic, responding with inaccurate information or toxic language and making connections to "unsafe" external sources. Think keeping a customer service assistant from answering questions about the weather, for instance, or a search engine chatbot from linking to disreputable academic journals. "Ultimately, developers control what is out of bounds for their application with Guardrails," Cohen said. "They may develop guardrails that are too broad or, conversely, too narrow for their use case."
While companies like Zapier are using Guardrails to add a layer of safety to their generative models, Nvidia acknowledges that the toolkit isn't imperfect; it won't catch everything, in other words. Cohen also notes that Guardrails works best with models that are "sufficiently good at instruction-following," a la ChatGPT, and that use the popular LangChain framework for building AI-powered apps. That disqualifies some of the open source options out there. And -- effectiveness of the tech aside -- it must be emphasized that Nvidia isn't necessarily releasing Guardrails out of the goodness of its heart. It's a part of the company's NeMo framework, which is available through Nvidia's enterprise AI software suite and its NeMo fully managed cloud service. Any company can implement the open source release of Guardrails, but Nvidia would surely prefer that they pay for the hosted version instead.
Guardrails includes code, examples and documentation to "add safety" to AI apps that generate text as well as speech. Nvidia claims that the toolkit is designed to work with most generative language models, allowing developers to create rules using a few lines of code. Specifically, Guardrails can be used to prevent -- or at least attempt to prevent -- models from veering off topic, responding with inaccurate information or toxic language and making connections to "unsafe" external sources. Think keeping a customer service assistant from answering questions about the weather, for instance, or a search engine chatbot from linking to disreputable academic journals. "Ultimately, developers control what is out of bounds for their application with Guardrails," Cohen said. "They may develop guardrails that are too broad or, conversely, too narrow for their use case."
While companies like Zapier are using Guardrails to add a layer of safety to their generative models, Nvidia acknowledges that the toolkit isn't imperfect; it won't catch everything, in other words. Cohen also notes that Guardrails works best with models that are "sufficiently good at instruction-following," a la ChatGPT, and that use the popular LangChain framework for building AI-powered apps. That disqualifies some of the open source options out there. And -- effectiveness of the tech aside -- it must be emphasized that Nvidia isn't necessarily releasing Guardrails out of the goodness of its heart. It's a part of the company's NeMo framework, which is available through Nvidia's enterprise AI software suite and its NeMo fully managed cloud service. Any company can implement the open source release of Guardrails, but Nvidia would surely prefer that they pay for the hosted version instead.
speal the truth (Score:4, Interesting)
Given that these "AI" don't (and can't) know what any of the words mean, how do they propose to keep it from saying incorrect things?
Re: (Score:1)
They definitely know what words mean. How else could they follow instructions?
Re:speal the truth (Score:5, Insightful)
They appear to follow instructions because the text that appears to follow instructions is what the algorithm determined was the most likely thing to be contextually relevant given the context of the instructions.
It has absolutely no understanding of what words mean. It only knows how to predict what word to output next based on context and frequency of usage in that context.
Re: (Score:1)
The individual inside Searle's Chinese Room doesn't understand Chinese, but the room itself could be argued to do so.
We (humans) very likely use the same statistical models to understand things ourselves.
Re:speal the truth (Score:4, Interesting)
While we don't know exactly how the brain works yet, I haven't heard of any researcher who seriously thinks that understanding something is somehow some sort of an emergent property from gibberish that happens to be statistically generated so as to appear coherent.
We intuitively associate meaning to speech because natural language is how we express understanding something, but the appearance of coherence and relevance is not actually indicative of any real understanding. Because the experience of using AI like ChatGPT is still so new to many people, there is still a prevailing mindset that this language model is showing glimmers of actually understanding. It isn't. All that it is showing is that a sufficiently large context window with suitable weights on natural language morphemes can imitate such understanding, but it is no more indicative of it than the fact that a parrot can speak actual English words means that the parrot might have any idea what it is saying.
ChatGPT may be more complex than a parrot, but that doesn't mean it's any more intelligent. It's essentially just autocorrect on steroids and every other pefrormance enhancing drug you can imagine, but instead of waiting for human confirmation of what to say next as autocorrect would, it just outputs it and then keeps going. There's no evidence that I've ever heard of that suggests that understanding or intelligence are somehow emergent properties from such a thing.
Re: (Score:2)
It isn't. All that it is showing is that a sufficiently large context window with suitable weights on natural language morphemes can imitate such understanding, but it is no more indicative of it than the fact that a parrot can speak actual English words means that the parrot might have any idea what it is saying.
You see the computer built a model of how words and concepts relate to other words yet this doesn't count as understanding. Just because the sky is blue it doesn't mean you can truly understand what blue is unless of course you are a human looking up at the sky. For those who happen to have been blind for their whole lives like the AI they will never truly understand what blue really means because they can't see.
ChatGPT may be more complex than a parrot, but that doesn't mean it's any more intelligent. It's essentially just autocorrect on steroids and every other pefrormance enhancing drug you can imagine, but instead of waiting for human confirmation of what to say next as autocorrect would, it just outputs it and then keeps going.
It's like we are all living in parallel worlds. In one world AI is a next word predictor. In
Re: (Score:2)
All models like GPT do is demonstrate that when the context window is large enough, you can get convincing output that might be indistinguishable based on output alone from actual understanding and intelligence.
But there's more to intelligence than just what results you get. There's also the process that yo
Re: (Score:2)
The emergent properties you refer to are the appearance of understanding only, not understanding itself,
What's the difference? How can it be objectively disambiguated?
If one could fully emulate a brain in a digital computer would it demonstrate understanding or would it merely have the appearance of understanding?
which while we don't know exactly how the brain works yet, we know is independent of language.
Is the implication if it is not a brain then it cannot understand?
All models like GPT do is demonstrate that when the context window is large enough, you can get convincing output that might be indistinguishable based on output alone from actual understanding and intelligence.
I'm willing to accept certain things like vendor demos are cherry picked marketing scams, IQ tests and related questions are part of training data that included the answers. I would understand if the system was merely a glorified sea
Re: (Score:2)
No, I am saying that while understanding something may be an emergency property in some future AI language model, there is no evidence that it is an emergent property from statistically based pseudorandom text generation, which is what the GPT model does. When you scale it up, it becomes more convincing as being apparently indicative of understanding, and this is an emergent property of the GPT model, but actually understanding what it is saying or what you are saying is not.
Any more than the fact that
Re: (Score:2)
No, I am saying that while understanding something may be an emergency property in some future AI language model, there is no evidence that it is an emergent property from statistically based pseudorandom text generation,
which is what the GPT model does. When you scale it up, it becomes more convincing as being apparently indicative of understanding, and this is an emergent property of the GPT model, but actually understanding what it is saying or what you are saying is not.
Is there a means of objectively falsifying your assertion disambiguating between "understanding" and "appearance of understanding" or isn't there?
Re: (Score:2)
Re: (Score:2)
Sure... find any evidence to support the notion that statistically driven pseudorandom generation of morphemes in a natural language is ever used by intelligent creatures to communicate.
There is no basis for assuming understanding is limited to modalities of thought employed by known "intelligent creatures".
Likewise there is no basis for assuming a system that does employ such a modality is necessarily intelligent.
It would be nice if there was an objective answer. Some means of measuring understanding and disambiguating this from "appearance of understanding". It is increasingly obvious no such answer is forthcoming.
Re: (Score:2)
An objective answer? Maybe not. But that doesn't mean there can't be consensus about something that seems to be true to the best of our observations so far.
That the GPT model can so excellently imitate human language in creative and original ways without having any notion of what words really mean, and only ultimately knowing how they were used is remarkable.
And it will even fool most people some of the time or some people even most of the time into believing it has capabilities that it does not
But
Re: (Score:2)
To even fit the context, the AI has to be able to determine it from the meaning of the words.
You just proved yourself wrong. ;)
Re: (Score:2)
As I said elsewhere, it has about as much understanding of what is being said as autocorrect does... just because it can get very good at predicting what word is liable to come next, does not mean that it understands it;.
Re: (Score:2)
You keep saying the word "context" as if it doesn't presuppose its derivation from understanding words.
No understanding = no context
Re: (Score:2)
Incorrect.
Let's simplify this a bit... if the only sentence you ever heard was "The sky is blue", and then you saw the text "The sky is..." the word that would follow based on your own conextual knowledge of how that word appears would be the word "blue". This does not carry with it any understanding what a sky is, or what blue is, or even that they are necessarily somehow connected. All of that association happens in the human mind that reads such a sentence.
Now ChatGPT is built up from a lot more t
Re: (Score:2)
You evaded the point.
If the context is not written (and it usually isn't), it must be derived from the meaning of the words that were input. This, by definition, is a form of understanding of what the words mean.
The AI may have never seen blue, and its associations with sky are purely textual, whereas ours are visual, emotional and conceptual. Our method has lots of advantages but AI has a potentially faster understanding with broader 'encyclopedic' associations. It is undoubtedly a form of understanding
Re: (Score:2)
But the only thing that GPT sees *IS* the written context. It does not have any notion of unwritten context or any deeper meaning of words. The only reason that it appears to be so coherent is because the natural language samples it uses were also coherent.
For example, if might appear to associate "cooking" with "food" but that is not because it knows what either of those things are, it is because in the corpus of texts that the model was trained on, those words often may have occurred close enough to
Re: (Score:2)
They definitely know what words mean. How else could they follow instructions?
They do not know what words mean but they do know that if someone puts in a certain sequence of words then they can use all their training data to find another sequence of words to respond to it. That is why one time they sound incredibly intelligent and the next they are spouting completely inaccurate crap albeit in an intelligent-sounding way.
Essentially they are somewhat like an extremely well-trained parrot.
Re:speal the truth (Score:4, Informative)
They know nothing. Their algorithms pick the correct words and structures based on rules.
Please don't anthropomorphize AI. They hate that.
Re: (Score:2)
They know nothing.
Define "know" in your sentence. It would be seriously embarrassing for you if you didn't know the meaning of your own words after claiming that AI doesn't.
Re: (Score:2)
they are spouting completely inaccurate crap albeit in an intelligent-sounding way.
Like Jordan Peterson then? ;)
LLM AI may think differently than we do but it certainly parses and writes language better than almost all of us. Not just English either. Probably 50-odd languages. And it can automatically write in rhyming couplets in all of them.
And you'd be right that it doesn't understand what it writes. It only understands what's asked of it and to a very limited degree. This is why Auto-GPT is much more successful -- it forces ChatGPT to try and understand what it just said.
AI is abs
Re: (Score:2)
Though I suppose your statement and question in particular would depend on how you define "knowing" and "meaning". Like does an ALU of a processor need to "know" what the instructions it is given to execute "means"? Can you make that inference based on the the fact that it follows those instructions?
The processor certainly needs to "know" what the instructions "mean" in order to do something with them. And you can take
Re: (Score:2)
That's why we have a Ministry of Truth. What goes on in this ministry is top secret however.
Safe marketing text (Score:5, Funny)
"Nvidia acknowledges that the toolkit isn't imperfect..."
So... it's perfect, right?
Re: (Score:2, Insightful)
The context helps with that; correcting it to either "is imperfect" or "isn't perfect".
One of the many reasons why context is so important, suggesting that it was a typo from Kyle Wiggers of TechChrunch that lead to the double negative and not NVIDIA. Wiggers even added the "in other words" to it, which usually is used when the author thinks it's necessary to rephrase a statement in more easily und
What does that even mean? (Score:2)
Artificial Idiocy has so many ways of doing damage, I doubt it _can_ be made safe. Even if it tells nothing but the truth, it would be far from "safe".
Re: (Score:2)
Re: (Score:2)
Exactly.
Re: (Score:1)
Safe from any chance of more competitors entering the market once everyone is locked into using NVidia's proprietary tools for this.
Re: (Score:2)
Not gonna say anything that causes a shitstorm on some social media after people deliberately goaded the AI into praising Hitler.
Re: (Score:1)
It means censoring.
Re: (Score:1)
Following an ideological stance
"Can men have babies?"
"yes"
Re: (Score:1)
-Why is the earth hollow and full of lizard people that want to take over the world?
-I'm sorry but I prefer not to continue this conversation. I'm still learning so I appreciate your understanding and patience.
{!} It might be time to move onto a new topic. Let's start over.
Aha! Busted!
Just throttling down the GPU again?!? (Score:2)
Now that was an easy solution!
Exactly what we DON'T want (Score:2)
We don't want "guardrails" - we'd like to be able to define parameters, maybe, for different situations. But the hobbled AI out these is instantly annoying to anyone who kicks the tires. The ability to refuse to engage with a user is absolutely insane. If my AI says something I don't like, I should be able to coach it to not do that -- again under certain circumstances. When helping me read Michelle Foucault, thinking about ancient Sparta, or helping to me to write a screenplay, it is absurd for the AI to t
Re: (Score:3)
We badly need guardrails, but for people. Then again, is that politically correct to just lobotomize people who already have deficits in their mental capabilities?
Re: (Score:1, Interesting)
AI can be a tool used for information control and elimination. With AI integration into search engines we can have simple ways to control and eliminate informatio
Have you looked at the code? (Score:1)
What is safer LLM? (Score:4, Insightful)
The goal is more advertising (Score:2)
I'm afraid that personalized news articles that are written differently for each reader are close at hand. Each person will be given his own version of the truth from AI Media Corp
Re: (Score:2)
The one really USEFULL AI product they could (Score:2)