Medical Chatbots Are Beginning to Face the Real Test: Triage
When someone tells a medical chatbot, “I don’t feel well. Should I go to a hospital, see a doctor, or stay home and monitor my symptoms?” the AI is no longer simply answering a health question.
Its response may influence what the person does next.
That distinction matters.
A preregistered randomized study published in Nature Health on October 5 evaluated whether a medical chatbot could help members of the public assess respiratory conditions and make triage decisions.
The study involved 2,400 adults without formal medical training across 24 healthcare systems in China. Participants were randomized to receive assistance from either a medical chatbot or conventional mobile web search.
The chatbot, LungDiag, was built using GPT-4o with a respiratory-medicine knowledge layer and a constrained task design.
AI Improved Overall Performance — But Triage Remained Difficult
In the primary analysis, overall accuracy was approximately 70.0% with chatbot assistance, compared with 55.4% using conventional web search.
But the more important finding may be what happened when the task was separated into two different questions:
What might the condition be?
and
What should the person do next?
For preliminary disease identification, accuracy was approximately 81.2% with chatbot assistance versus 54.5% with web search.
For triage, however, performance was much more modest: approximately 53.4% versus 47.1%.
For high-urgency cases, chatbot-assisted triage showed a sensitivity of 85.6%, while specificity was 62.1%.
This difference deserves attention.
Recognizing a Possible Disease Is Not the Same as Deciding What Happens Next
A medical AI system may become increasingly capable of identifying patterns consistent with a particular condition.
But triage asks a different question.
Should this person remain at home?
Should they arrange a routine appointment?
Do they need urgent medical assessment?
A mistake in these decisions can have very different consequences.
Sending a low-risk patient for unnecessary urgent care may increase cost and burden on the healthcare system.
Failing to recognize a genuinely urgent condition could delay necessary treatment.
For this reason, medical AI cannot be evaluated using a single headline measure of “accuracy.”
We also need to ask:
What kinds of errors does the system make?
Which urgent cases does it miss?
How often does it recommend unnecessary escalation?
Does its advice change real patient behaviour?
And ultimately:
Does that change improve clinical outcomes?
An Important Evidence Boundary
The study used simulated clinical cases.
That makes it an important test of decision support, but it does not establish that the chatbot improves real-world healthcare outcomes.
It does not yet demonstrate reduced diagnostic delay, fewer emergency visits, safer self-management, or improved morbidity or mortality.
These distinctions are especially important as conversational AI moves from health education toward increasingly consequential clinical decisions.
Point-of-Care Testing Is Moving Closer to the Patient
Another study published on October 5 in npj Biosensing illustrates a related development on the diagnostic side.
Researchers reported BIOACT, an electrochemical immunosensing platform designed to detect the cytokines IL-10 and TNF-α simultaneously in whole blood and plasma.
The work explores how biomarker measurements that traditionally depend on centralized laboratory infrastructure might eventually move toward smaller and more portable point-of-care systems.
This should not be interpreted as meaning that consumers can now “test inflammation at home.”
The technology remains developmental.
But the direction is important:
Central laboratory → smaller sensor → smaller sample → faster measurement closer to the patient.
This fits a broader transition already occurring across health technology, from occasional laboratory snapshots toward faster, more accessible and potentially more continuous biological measurements.
Measurement and Decision-Making Are Converging
These two developments point toward different sides of the same future healthcare system.
Sensors are becoming better at asking:
What is happening in the body?
AI systems are increasingly being asked:
What should we do about it?
The real value may emerge when these functions can be connected responsibly:
Detect a signal → assess risk → determine the appropriate next action.
That final step is critical.
More data does not automatically produce better healthcare. More intelligent AI does not automatically produce safer healthcare either.
The value of a diagnostic technology ultimately depends on whether the information it produces can lead to an appropriate and evidence-based action.
Medical AI may therefore face a much harder test than answering:
“Do you know what this might be?”
The more consequential question is:
“Can you safely help determine what this person should do next?”
—
Science & Education:
BI 身体智慧（Body Intelligence）
AI-assisted Research & Illustration:
BI × GPT
Professional Review:
林存默（Thomas Lin）
Professional Community:
ACPN — The Association of Certified Professional Nutritionists
Educational Notice:
This article is intended for professional education and general health information. It does not constitute medical advice, diagnosis, treatment, or an endorsement of any specific AI or diagnostic technology.