into a lot of the medical services now. Private medical practitioners, Amazon's One-Medical, and most hospitals use them to relieve staff of having to take notes. Even the HBO show the Pitt has a season story line about how the transcription errors take more time to correct than taking notes in the first place.
I am willing to bet that in many cases patients inadvertently agreed to allow AI transcription. The tablets handed out in waiting rooms before treatment DO NOT LET YOU EDIT THE AGREEMENTS. And if you refuse to sign, you do not get treatment.
I've also noted the manipulative language (if you let AI transcribe and share everywhere you get an imaginary pony, if not you get real shit and make your health provider unhappy and overworked) and "dark pattern" selection tricks.
Suggested books on these topics "The AI Con" and "Empire of AI".
A search engine (without AI) or Wikipedia are more reliable tools to collect accurate information that does not have to be cross-checked.
By the way... here is an LLM AI answer to the question: Why do LLMs hallucinate?
The Answer:
LLMs hallucinate because they're fundamentally **probabilistic pattern-matchers**, not truth-seekers. Here's why:
## The Core Problem: Pattern vs. Reality
**LLMs don't "know" things the way humans do.** They:
- Analyze trillions of text patterns from training data
- Predict what word comes next based on statistical likelihood
- Generate responses by combining these patterns into coherent-sounding text
But this means they can confidently produce **plausible-sounding falsehoods** because:
- They've seen the words "apple" + "fruit" together 10 million times "apple is a vegetable" sounds like it could be true to their training
- They've never actually "experienced" or "verified" anything
## Why Hallucinations Happen
### 1. **Training Data Issues**
- LLMs are trained on massive datasets that include misinformation (misleading articles, fake news, outdated info)
- They learn to associate words regardless of truth — "NASA has a Mars colony" may seem reasonable if it's been mentioned in sci-fi novels
### 2. **Confidence Without Verification**
- LLMs generate responses with high confidence levels (because that's what their training taught them)
- But confidence accuracy
- They're essentially "very convincing guessers" rather than fact-checkers
### 3. **Lack of Real-World Experience**
- No ability to verify claims through experimentation or observation
- Can't distinguish between:
- "The Earth is round" (true)
- "The Earth is flat" (false, but plausible in context)
## The "Hallucination" Mechanism
Think of it like this:
```
Input: "What's the capital of France?"
LLM: "Paris" (because it sees that pattern 100x more than other cities)
Input: "Who discovered penicillin?"
LLM: "Alexander Fleming" (true)
But if asked about a made-up scientist:
LLM might confidently say "Dr. Smithson" and sound completely reasonable
```
## Real-World Examples
### Medical Hallucinations:
> **User**: "What's the treatment for diabetes?"
> **LLM Response**: "Take 100mg of Xanax daily" (confidently, but wrong)
### Historical Facts:
> **User**: "When did World War II end?"
> **LLM Response**: "1945" (correct)
> But then: "The war was won by the United States and Japan" (incorrect but plausible)
## The Fundamental Limitation
LLMs are essentially **statistical approximators** — not truth machines. They:
- Learn to mimic human conversation patterns
- Generate text that sounds like it could be true
- Have no internal mechanism for determining what is actually true vs. false
This is why:
- They're great for creative writing, summarization, and general conversation
- But dangerous for critical decision-making (medical advice, legal contracts, scientific research)
## The Solution (Not Fully Resolved Yet)
Currently, we have to:
1. **Verify everything** they say (even if it sounds right)
2. **Cross-check with trusted sources**
3. **Be transparent about uncertainties**
LLMs are powerful tools — but they're not replacements for human judgment, verification, or expertise.
---
> **Bottom Line**: LLM hallucinations happen because they're trained on text patterns rather than reality, and they're optimized to be persuasive, not accurate. They're like a very smart but unfiltered Wikipedia — useful, but potentially dangerous without fact-checking.
