Patients Aren't Being Told About the AI Systems Advising Their Care (statnews.com) 42
At a growing number of prominent hospitals and clinics around the country, clinicians are turning to AI-powered decision support tools -- many of them unproven -- to help predict whether hospitalized patients are likely to develop complications or deteriorate, whether they're at risk of readmission, and whether they're likely to die soon. But these patients and their family members are often not informed about or asked to consent to the use of these tools in their care, a STAT examination has found. From a report: The result: Machines that are completely invisible to patients are increasingly guiding decision-making in the clinic. Hospitals and clinicians "are operating under the assumption that you do not disclose, and that's not really something that has been defended or really thought about," Harvard Law School professor Glenn Cohen said. Cohen is the author of one of only a few articles examining the issue, which has received surprisingly scant attention in the medical literature even as research about AI and machine learning proliferates. In some cases, there's little room for harm: Patients may not need to know about an AI system that's nudging their doctor to move up an MRI scan by a day, like the one deployed by M Health Fairview, or to be more thoughtful, such as with algorithms meant to encourage clinicians to broach end-of-life conversations.
But in other cases, lack of disclosure means that patients may never know what happened if an AI model makes a faulty recommendation that is part of the reason they are denied needed care or undergo an unnecessary, costly, or even harmful intervention. That's a real risk, because some of these AI models are fraught with bias, and even those that have been demonstrated to be accurate largely haven't yet been shown to improve patient outcomes. Some hospitals don't share data on how well the systems work, justifying the decision on the grounds that they are not conducting research. But that means that patients are not only being denied information about whether the tools are being used in their care, but also about whether the tools are actually helping them. The decision not to mention these systems to patients is the product of an emerging consensus among doctors, hospital executives, developers, and system architects, who see little value -- but plenty of downside -- in raising the subject.
But in other cases, lack of disclosure means that patients may never know what happened if an AI model makes a faulty recommendation that is part of the reason they are denied needed care or undergo an unnecessary, costly, or even harmful intervention. That's a real risk, because some of these AI models are fraught with bias, and even those that have been demonstrated to be accurate largely haven't yet been shown to improve patient outcomes. Some hospitals don't share data on how well the systems work, justifying the decision on the grounds that they are not conducting research. But that means that patients are not only being denied information about whether the tools are being used in their care, but also about whether the tools are actually helping them. The decision not to mention these systems to patients is the product of an emerging consensus among doctors, hospital executives, developers, and system architects, who see little value -- but plenty of downside -- in raising the subject.
AI To the Rescue (Score:2)
Alternative? (Score:3)
A resident looking up your malady in a text books and then guessing?
As long as the AI is used to augment the judgement of an experienced physician, then there is no issue.
Re: Alternative? (Score:3)
Its based on statistics. So as long as your a textbook case you should be ok. Im never a textbook case so Im a little worried. But then again I challenge and confirm everything Im told anyway.
You know the saying
There are Lies. Damn Lies. And Statistics.
Re: (Score:3)
Expecting personalized care in the revolving door of hospital staff is not a good bet. You have to be your own advocate.
No, I cannot take that medication and eat twenty minutes later. I repeated that sentence so much. It was also a fight to get the right care.
I swear I want to hire a private physician to handle the hospital next time I have a big ordeal. Kinda like how you get a contractor to manage your housing development.
Re: (Score:1)
Seems fine to me (Score:5, Insightful)
As long as they don't take the FB defense (we're not responsible, the algorithms are), this seems fine to me. Doctors don't regularly inform you of the journals they're reading or how a PCR test works. That's their job. It's their concern.
Re: (Score:2)
the key here is "proven". deep learning and statistical analysis can actually outperform humans which are error prone too and subject to bias.
the algorithms must be proven to be free of bias and at least just as accurate as human diagnose on average. if they are there is no problem, even the more the merrier. i don't see why a specific consent should be necessary, you already consent a great deal when you seek healthcare. if they aren't they shouldn't be used plain and simple, not even with consent.
Re: (Score:2)
funny that the mere word "bias" triggered you into such a rant about ... race?
way to miss a point. it actually refers to a fundamental factor in every form of statistical analysis. now that you know, feel free to reread the post calmly so you can actually have a clue what we are talking about. otherwise have fun with your incoherent rambling.
MYCIN, back in the 1970s (Score:2)
Analyzed bacterial infections using a rule based system. People were concerned about the mistakes that it made, and it did indeed make mistakes.
But in trials it made fewer mistakes and produced better results on average than doctors in the field.
Things do not have to be perfect to be good.
It never took off because in practice it usually recommended what doctors would do anyway, and entering all the test results and observations into a 1970s teletype was tedious.
Being a rule based system it could also provi
Re: (Score:2)
rule based systems are very hard to "train" (you don't, you actually painstakingly write them) and thus are inherently limited in scope, scale and domain. they are still useful at the very specific level. current systems use a mix of techniques including rule based inference.
anyhow the process and implementation is not really relevant here, it's the accuracy that matters and you can test that with any black box. one of the recent shifts in ai is that we got used to stuff that works even if we often don't ex
Re: (Score:2)
People are rightfully wary of black boxes results from either computers or people. A reasoned argument tells them what has and has not been considered.
There is more to AI than just plugging everything into a magic artificial neural network and letting it learn. ANNs are only one tool in the box.
Re: (Score:2)
People are rightfully wary of black boxes results from either computers or people. A reasoned argument tells them what has and has not been considered.
yet people have been eating all sorts of food for millenia without any reasoned argument for it except: "it works". actually, we are still pretty clueless about the finer details on how it works, we just keep eating it because it does.
There is more to AI than just plugging everything into a magic artificial neural network and letting it learn. ANNs are only one tool in the box.
nobody said it shouldn't. my point was precisely "proper verification" and "right questions asked". and i didn't even mention neural networks nor any other tool, you singled out rule based inference, implying it is superior because you know exactly how it works. well, it isn't
Re: (Score:2)
i hope next time you need some diagnostic you can have your choice.
just realized this could have a pretty nasty interpretation. well, it wasn't meant that way :)
Re: Seems fine to me (Score:2)
Doctors prescribe off-label all the time. That's pretty much the definition of unproven medicine, but we seem to do alright with it.
The ultimate liability dodge (Score:2)
Re: (Score:3)
Re: (Score:2)
The trick to hospitals is you need to shank one or the doctors your first day there. Then everyone in the hospital will respect you.
Tools (Score:4, Insightful)
In other news, patients also weren't asked for their informed consent about doctor's using stethoscopes, blood pressure cuffs, small rubber mallets, EKG machines, or any other tool in their arsenal.
Nor were they informed about the doctor's use of reference material, consultative opinions, or the fax machine.
Re: (Score:2)
or the fax machine.
Is anyone else impressed that AI is being used for medical diagnosis in an industry where medical professionals spend 30 minutes two finger typing a short message into a computer, printing it, and then faxing the result into the ether?
Overblown (Score:5, Insightful)
Why? Because ultimately it is your name on the line making life and death recommendations.
Well, there's some good news and some bad news (Score:2)
"Ok, doctor, give me the good news first."
"We have an artificial intelligence advising us on your condition, and we have a recommendation on how to proceed."
"And the bad news? Give it to me straight."
"The bad news is that I can't give it to you straight. Well, the AI could, but neither you nor I could understand it [youtube.com]. You know, because, AI."
Re: (Score:2)
ECC errors led to make enhancement therapy to treat your skin rash.
How far should disclosure go? (Score:2)
Do patients get a complete list of staff, every single person involved in their care and decision making and patient care, complete list of books which the doctor might consult (or maybe even a complete list of books they read), a complete list of past patients which obviously will affect the doctors decisions as they comprise the doctor's "work experience". From doctors which might have consulted on the case over lunch with their primary physician, through all the nurses and lab techs who might have offere
Re: (Score:2)
That sounds like customers of pretty much every business in the known universe. You should hear the stories my wife tells after 25 years in retail.
At very least I should know about out of network (Score:2)
At very least I should know about out of network people
How to increase health outcomes: (Score:2)
Ask patients to decide their own care and their own tools.
Better still let's ask a homeless person in the street, I'm sure they know as much about medicine as anyone else who goes to see a doctor.
Re: (Score:2)
they will under trumpcare find out when blackliste (Score:1)
they will under trumpcare find out when blacklisted
And what about the non-AI systems? (Score:2)
Is this a joke? (Score:1)
Do you want the Butlerian Jihad? (Score:2)
Because this is how you get the Butlerian Jihad.
I refer to the one described in the Dune Encyclopedia.
Re: Do you want the Butlerian Jihad? (Score:2)
Is there another Butlerian Jihad I don't know about?
Everything I know, I learned from the Orange Catholic Bible.
Re: (Score:2)
Is there another Butlerian Jihad I don't know about?
(hastily) No. No, there is not.
It's a classic way to avoid the issue (Score:2)
If there are known issues, they need to be solved. A patient could sign a waver and the doctors will be happy, or the patient could decide (without any knowledge to back it up) that human bias is better than AI bias. All that does is reduce doctor liability. It makes things worse, not better, because instead of making sure that the AI does its work well (that is, at least comparably to an average doctor), it makes sure that potentially helpful AI might not be used and bad AI will not be eliminated because t