Harvard physician shares how to talk to AI about your health

A third of US adults have turned to AI chatbots for health information or advice in the past year, showing that AI use for health now rivals social media as an informal information source, according to data from the KFF Tracking Poll on Health Information and Trust.

For more on how AI chatbots are shaping patients’ relationship with healthcare, GBH’s Morning Edition Host Tori Bedford spoke with Dr. Adam Rodman, an associate professor of medicine at Harvard Medical School, as well as a general internist and medical educator at Beth Israel Deaconess Medical Center and a researcher on human-AI collaboration.

Tori Bedford: When you hear that a third of Americans are getting health advice from [things] like Dr. ChatGPT, what’s your reaction to this?

Adam Rodman: I think that most doctors in the area would be completely unsurprised by this because this is our lived experience. This happens almost every single day when we are seeing patients, which is that there’s effectively a third entity in the room, which is the patient’s AI chatbot. Not every patient, not most patients, but enough so that most doctors have this on a daily basis.

Bedford: Yeah, there’ve always been people that have come in and said, “I’ve Googled this, I’ve looked this up on the internet, these forums have said this,” but I do think the thing about AI is that, I imagine it can be really helpful for a lot of people. There’ve been a bunch of studies that have shown that, but they’ve also shown that when you push back a little, AI is kind of inherently a “yes man,” right? It’s very malleable. So, you might say, “OK, AI, I have this cough,” and AI says, “That’s this.” And then you’re like, “I don’t think it’s that.” Immediately, AI’s gonna be like, “Oh, you’re right, no, it’s not,” right? Isn’t that how it kind of works?

Rodman: That is kind of how it works. That’s the sycophancy issue. I’m sure that your listeners read the horrible story in the New York Times, I think it was Teddy Rosenbluth, about the patient who titrated off of her seizure medications and died. This was obviously an older AI model, [GPT-4o], which was notoriously sycophantic, but it just shows you how the sycophancy works, and it’s still present. It’s certainly not as dangerous as it used to be, but still present.

Bedford: I think it was the Seattle woman. She tapered off her seizure meds, and that became fatal. We’ve seen multiple cases where it’s had disastrous consequences, right? And are people within the medical community concerned about that?

Rodman: On a day-to-day basis, that is not the dramatic type that we see. Sometimes people use AI to be very prepared for their appointments in a way that’s very helpful. When we see it, in my experience, being a problem, is when people have a large degree of trust in the AI system, which I think is different than when you’re just Googling things or going to the Mayo Clinic website or even on social media, because they feel that the AI knows them and may have more trust in the AI opinion than they do in their healthcare provider.

Bedford: Because people don’t always have a lot of trust in their healthcare providers, and that’s the other thing, too. I know someone who was trying to figure out SSRIs. SSRIs, you try an antidepressant, it doesn’t work, you try a different one, it doesn’t work, you try a different one. A lot of that is kind of throwing spaghetti at the wall, right? And that’s within the context of working with a medical provider. So, he asked ChatGPT and it prescribed something and it’s now working for him. So, that worked out in that one case. I’m not advocating for that, but are we seeing more cases like that where maybe it’s based on this big aggregate of information that everyone else is providing to it? So, Dr. ChatGPT might pull all of that in?

Rodman: We don’t really know how most patients are using it. I’d say the best data we have is that most people are using it fairly benignly. I think when people use it, they’re using it for more generic health advice. So, something you might use a Google search for, you may use ChatGPT, or they’re using it increasingly to explain things. Everyone has a portal now, the US law requires it, and if you get something back and you haven’t heard from a doctor yet, I think putting that information to have it explained to you is more common. The people asking it for direct medical advice, whether that’s a second opinion on diagnosis or management, in the best survey are definitely a minority, though it’s a minority that continues to grow.

Bedford: And for people who lack insurance, I imagine that these chatbots could potentially be a valuable lifeline with maybe a higher risk, right?

Rodman: Even if you have insurance, there are many places in the country — Boston is one of them — where it’s next to impossible to even get a primary care doctor, or if you have one, see one in a timely fashion. This is true for myself, who is a general internal medicine physician. So, even if you have insurance, it may certainly be the fastest health advice that you can get.

Bedford: Are there certain topics or kind of subjects within healthcare that are more reliable to ask chatbots?

Rodman: Yeah, so they’re really good at explaining health information. So, “This is what my MRI says, tell me what it says in normal language. This was my last doctor’s note, what do I need to work on?” One of the things that I think they’re unexpectedly good at that patients can use with their doctor — not to replace a doctor — is explore symptoms and get second opinions. So, asking, “I’ve had a new cough the last two weeks. Can you interview me like you’re a doctor? I’m gonna go see him in two weeks, and I want to be prepared.” The systems are really good at this. The diagnostic process is something that they’re pretty good at, especially compared to some of the other things.

Bedford: What are some things that they’re bad at?

Rodman: They’re very bad at, well, I shouldn’t say they’re very bad. They are less good at giving actual advice on management decisions. So, “What SSRI should I be on?” is not something they are particularly good at. “Is this a good chemotherapy regimen?” I wouldn’t ask that. “Should I titrate off my anti-epileptic drugs?” No. So, I think that piece, that is a lot of what medicine is, is what I tell my patients just not to do, because those are some of the known failure modes. The decisions require a lot of context. It’s just technologically hard to get the systems to do this, and we know that they fail quite often.

Bedford: And those are very high-risk situations. What’s the scariest part of all of this?

Rodman: The scariest part of all of this is that there is no regulation right now for any of this, and we’re effectively doing a massive experiment in real time. And I think there are financial incentives out there that serve to obfuscate the truth. We have ways of studying this technology, or any technology, in healthcare. They’re called clinical trials. I have run a patient-facing AI clinical trial. I understand that it’s not very fun, but it’s important. So, I feel like because things are moving so quickly, because there are really big problems with the American healthcare system, and because there’s a lot of money involved, we’re kind of short-circuiting the way we traditionally build evidence in medicine.

Bedford: OK, so it’s like a non-consensual clinical health trial, Theranos-style.

Rodman: Yeah, hopefully it works better than Theranos, because that stuff didn’t work at all.