Is there a doctor in the house?
Well, right there on your computer is AI. And health is one of the top reasons people turn to it.
AI can pass medical licensing exams, outperform doctors at reading electrocardiograms, and, AI-powered stethoscopes detect heart failure in seconds.
Maybe that’s why today one in six Americans turn to AI chatbots at least once a month in search of health advice.
But is the AI doctor ready to see you now?
That’s where things get complicated.
I had my own experience with AI. I tore cartilage in my thumb and the chatbot recommended high doses of amino acids for protein. Two weeks later, my kidney function started to falter.
But this was AI with vast stores of medical knowledge! What went wrong?'
A recent study on 1,300 people in the UK sheds some light. Each participant was given a made-up medical scenario — complete with symptoms and lifestyle details — to use to consult a chatbot.
The result: AI did no better at health decisions than ordinary internet searches like Google.
But when researchers tested the same scenarios without humans in the loop, AI identified the medical condition correctly nearly 95 percent of the time.
The humans, it turns out, didn’t know how to describe symptoms. They left out critical information about the health problem and about themselves. Sometimes they just didn’t use the right words. In one case, the simulated patient described the “worst headache ever” and was told by Chat to go to the hospital. Another participant said “terrible headache” and was told to lie down in a dark room, even though it was the same life-threatening situation.
It’s doctors who are trained to know how to elicit information from you - and they know you. Doctors constantly pivot their diagnostic direction based on what they are hearing and observing. For them, a kind of informed intuition takes over.
In my case, I neglected to tell AI that I’m thin and petite. If I had spoken with my doctor - who knows me - she would have cautioned that someone my size shouldn’t take high doses of protein.
So what should you do if you decide to turn to AI for health advice? Your job is to help Chat help you. You can ask AI to interview you the way a doctor might, and to continue drilling down with follow-up questions. Also, keep asking what information it needs. You can also ask it to critique its own opinion.
The makers of ChatGPT routinely update their models with these things in mind. But they advise users not to think of Chat as a doctor, but rather a tool to help you understand test results, and weigh options for when you do get to a real live doctor, who, by the way, often consults AI for tricky cases.
The truth is, we’re still in the infancy of medical AI.
When GPS first came out, a driver followed the directions and drove into a swamp. Another, into a lake.
So be careful.
Don’t let the AI doctor entirely take over the wheel.
