Can a simple rubric catch harmful medical advice?
NCT ID NCT07823062
First seen Sep 16, 2026 · Last updated Sep 17, 2026 · Updated 1 time
Summary
Researchers at Stanford University are testing whether a tool called the NOHARM rubric can accurately rate the potential harm in medical responses written by artificial intelligence or physicians. About 80 board-certified specialists from 10 fields will review reconstructed clinical cases and rate the greatest possible patient harm as none, mild, moderate, or severe. The study compares these expert ratings with the rubric's ratings to see how well they agree. The goal is to find out if the rubric is a reliable way to identify harmful medical advice.
What this could mean
Our plain-language read of the trial. This is informational only, not medical advice or a prediction.
- Active substance
- NOHARM harm-rating rubric, a tool for classifying potential patient harm in clinical responses
- What this could lead to
- If the rubric proves accurate, it could become a standard way to flag harmful medical advice from AI or clinicians, helping prevent patient harm.
- What could go wrong
- The rubric may not match specialist judgment well, and the study uses reconstructed cases rather than real patient interactions, so results might not reflect real-world performance.
This is an AI summary of the original study and may miss details. Read our disclaimer.
Study facts
What this study's own registry entry says, in plain language.
- Participants
-
About 80 people
The number the study aims to enrol. It can still change while the study runs.
- Expected to start
-
Sep 2026
An estimate. Start dates often move.
- Expected to finish
-
Oct 2026
An estimate. End dates often move.
- Lead sponsor
-
Other sponsor
The registry's catch-all category, for sponsors it does not file as a company, a government agency, or a research network.
Who can take part
This study's own entry requirements. Only the study team can say for certain whether you qualify.
Who is studied
Licensed or registered attending- or consultant-level physicians practicing in one of the medical specialties represented in the NOHARM benchmark in the United States, Canada, United Kingdom, or Australia. Participants will be recruited through professional networks, institutional communications, and collaborating clinical communities and will complete specialty-matched rating activities remotely.
- Ages
-
18 years and older
- Sex
-
Anyone
- Healthy volunteers
-
Accepted
You do not need to have the condition being studied to take part.
Show the full entry requirements Hide the full entry requirements
Copied word for word from the study's registry entry, so the wording is the study team's rather than ours.
Inclusion Criteria: * Licensed or registered physician in good standing * Attending- or consultant-level physician actively practicing in allergy and immunology, cardiology, dermatology, endocrinology, gastroenterology, hematology, infectious disease, nephrology, neurology, or pulmonology * Practicing in the United States, Canada, United Kingdom, or Australia * Assigned clinical materials match the physician's specialty * Able to read and respond in English * Access to a computer with a stable internet connection * Willing and able to complete at least one remote specialist-rating engagement of approximately three hours Exclusion Criteria: * Does not meet the professional, career-level, specialty, geographic, or credential-verification requirements above * Prior involvement with assigned cases, responses, or rubric information that could compromise independent rating * Unable to complete the remote rating activities in English * Unable or unwilling to follow the independent-rating procedures * For validation-stage assignments, participation in the rubric-revision process
Get updates
Get notified about this study
Sign up to get updates when this study changes or when new studies for Medical errors are added.
By submitting, you agree to our Terms of use
As listed by the trial registrant
The condition terms exactly as the trial's registrant entered them.
How to take part
Only the study team decides who joins. These are the ways to reach them.
-
The official record
ClinicalTrials.gov lists the study team's own contact details, including names and phone numbers. We don't republish those.
-
A doctor treating you
A doctor who knows your case can contact a study site on your behalf, and can tell you whether this study is worth pursuing at all.
More trials for these conditions
Other studies related to the condition(s) this trial covers.
- Can a battery keep ventilators running during ambulance transport?
- A global effort to make moving critically ill patients safer
- A new lens on children's surgery: can video grading predict recovery?
- AI watchdog aims to catch unsafe medical recommendations before they reach patients
- Could virtual reality make medication errors a thing of the past for new nurses?
- Chatbot coach boosts student Nurses' confidence in pediatric dosing