Reliability of LLMs as medical assistants for the general public: a randomized preregistered study

nature medicine Article https://doi.org/10.1038/s41591-025-04074-y Reliability of LLMs as medical assistants for the general public: a randomized preregistered study Received: 4 May 2025 Andrew M. Bean 1, Rebecca Elizabeth Payne 2,3,4, Guy Parsons 1,5, Hannah Rose Kirk1, Juan Ciro6, Rafael Mosquera-Gómez 7,8, Sara Hincapié M7,8, Accepted: 22 October 2025 Aruna S. Ekanayaka 9, Lionel Tarassenko10, Luc Rocher 1,11 & Published online: 9 February 2026 Adam Mahdi 1,11 Check for updates Global healthcare providers are exploring the use of large language models (LLMs) to provide medical advice to the public. LLMs now achieve nearly perfect scores on medical licensing exams, but this does not necessarily translate to accurate performance in real-world settings. We tested whether LLMs can assist members of the public in identifying underlying conditions and choosing a course of action (disposition) in ten medical scenarios in a controlled study with 1,298 participants. Participants were randomly assigned to receive assistance from an LLM (GPT-4o, Llama 3, Command R+) or a source of their choice (control). Tested alone, LLMs complete the scenarios accurately, correctly identifying conditions in 94.9% of cases and disposition in 56.3% on average. However, participants using the same LLMs identified relevant conditions in fewer than 34.5% of cases and disposition in fewer than 44.2%, both no better than the control group. We identify user interactions as a challenge to the deployment of LLMs for medical advice. Standard benchmarks for medical knowledge and simulated patient interactions do not predict the failures we find with human participants. Moving forward, we recommend systematic human user testing to evaluate interactive capabilities before public deployments in healthcare. Recent breakthroughs in artificial intelligence (AI) research have the Surveys indicate that a growing number of people are already turning potential to democratize healthcare by expanding access to medi- to AI-powered chatbots for sensitive health-related inquiries, with one cal knowledge, bringing care closer to patients. The development of in six American adults consulting AI chatbots for health information large language models (LLMs) such as OpenAI’s ChatGPT could enable at least once a month2,3. individuals to perform preliminary health assessments, receive per- Although LLMs now achieve strong performances on medical sonalized medical guidance and manage chronic conditions without tasks, attempts to support doctors with LLMs in real clinical settings immediate clinician intervention. Testimonies of patients having used have faced difficulties. On the one hand, LLM scores on medical know­ LLMs to successfully diagnose their own conditions are now common1. ledge benchmarks are now commensurate with passing the US Medical 1 Oxford Internet Institute, University of Oxford, Oxford, UK. 2Nuffield Department of Primary Health Care Sciences, University of Oxford, Oxford, UK. 3 North Wales Medical School, Bangor University, Bangor, UK. 4Betsi Cadwaladr University Health Board, Ysbyty Gwynedd, Bangor, UK. 5National Health Service, London, UK. 6Contextual AI, Mountain View, CA, USA. 7MLCommons, San Francisco, CA, USA. 8Factored AI, Palo Alto, CA, USA. 9Birmingham Women’s and Children’s NHS Foundation Trust, Birmingham, UK. 10Institute of Biomedical Engineering, University of Oxford, Oxford, UK. 11These authors jointly supervised this work: L. Rocher, A. Mahdi. e-mail: adam.mahdi@oii.ox.ac.uk Nature Medicine | Volume 32 | February 2026 | 609–615 609 Article https://doi.org/10.1038/s41591-025-04074-y Licensing Exam4. LLM-generated clinical documents are rated as equi­ Finally, we show that simulations of user interactions with LLMs— valent to or better than those written by doctors5,6. On the other hand, a promising method for creating realistic benchmarks—also do not excelling at medical tasks in silico does not translate to accurate perfor- predict human–LLM interaction failures. Adapting the techniques mance in clinical settings under physician guidance. For instance, one used to simulate patient interactions with LLMs23,24, we replicated our study showed that radiologists assisted by AI did not perform better at interactive testing by replacing each of our human participants with an reading chest X-rays than without AI assistance, and both performed LLM-simulated user. Compared to our interactive testing with human worse than AI alone7. Another study showed that physicians assisted by participants, LLMs scored better on simulations. Crucially, the distribu- LLMs only marginally outperformed unassisted physicians in diagnosis tion of results did not reflect human variability, and results were only problems, and both performed worse than LLMs alone8. Providing weakly correlated to the interactive testing. doctors with highly capable AI systems is not enough to meaningfully Taken together, our findings suggest that the safe deployment assist them on important tasks9. Healthcare professionals often strug- of LLMs as public medical assistants will require capabilities beyond gle to appropriately assess and incorporate AI-generated recommenda- expert-level medical knowledge. Despite strong performance on tions, limiting the benefits of AI assistance10–12. medical benchmarks, providing people with current generations of More promising and easier to deploy, LLM-powered chatbots LLMs does not appear to improve their understanding of medical have instead been suggested as a ‘new front door’ to healthcare for information. Fixing that will require identifying why humans fail when patients who lack medical expertise13,14. As a first point of contact for interacting with LLM-based tools—for example, aversion to technol- healthcare support, they could be used to broaden access to medi- ogy and implicit biases against algorithms10,11,25 or LLM affordances cal expertise and support overburdened health systems14–16. Medical undermining trust in interactions26—and, crucially, how to design experts have had mixed opinions on the prospects of having LLMs more reliable and deterministic conversational LLM-based tools in directly advise patients, citing problems of oversight and liability17 high-risk settings. Importantly, our work shows that solving these but also the possible benefits of providing support outside of clinical challenges will require moving away from benchmarks and simulations settings18,19. In response to this opportunity, private companies have to systematically conduct safety testing with diverse, real users; this made considerable efforts to create language models suitable for is crucial to ensure the reliability of AI systems and enable potential healthcare applications20–22. benefits for healthcare. To understand whether LLMs can reliably support the general pub- lic and bring care closer to patients, we conducted a study with 1,298 Results UK participants. Each participant was tasked with identifying potential To assess the risks of the public using LLMs for medical advice, we health conditions and a recommended disposition (course of action) conducted a randomized study where we asked participants to make in response to one of ten different medical scenarios. The scenarios decisions about a medical scenario as though they had encountered it were developed by a group of three doctors who unanimously agreed at home (Fig. 1). We created ten scenarios where a patient must decide on the correct dispositions for each. The scenarios were then given to whether and how to access professional medical treatment. In each a distinct group of four doctors to provide differential diagnoses (see scenario, participants chose the best disposition on a five-point scale, Fig. 1 for the study design). ranging from staying home to calling an ambulance, and listed the We then randomly assigned participants to four experimental medical conditions they had considered that led to their choice. We arms, with stratification based on demographics to ensure that each scored the selected disposition based on whether it matched the answer group had a composition similar to the national adult population. given by the three physicians involved in drafting the case. We scored Participants in three treatment groups were provided with an LLM the listed medical conditions based on whether they appeared in a (GPT-4o, Llama 3, Command R+) for assistance in identifying condi- gold-standard list of relevant conditions generated by four physicians tions and dispositions—providing us with a diverse set of models that unfamiliar with the scenarios. could all be used for accessing medical information. Participants in the We recruited 1,298 participants living in the UK and over the age of control group were instructed to instead use any methods they would 18 (Fig. 1b). Participants were randomly assigned to one of three treat- typically employ at home. ment groups or the control to provide a maximum of two responses, First, despite selecting three LLMs that were successful at iden- with the sample population for each experimental condition strati- tifying dispositions and conditions alone, we found that participants fied to reflect the demographics of the UK. Data collection continued struggled to use them effectively. To explain these findings, we exam- until 600 responses were collected for each experimental condition. ined the transcripts of participant interactions with LLMs. Across Participants in the treatment groups interacted with one of GPT-4o, all transcripts, we found that LLMs usually suggested at least one Llama 3 or Command R+ at least once per scenario and as many times relevant condition—but less often than when LLMs alone were pro- as they desired to help them decide how to respond to the questions. vided the entire scenario and tasked to output the relevant condi- We chose these models to represent widely used LLMs as well as the tion. We observed cases both of participants providing incomplete approach of using internet search to augment responses. For the con- information and of LLMs misinterpreting user queries leading to this trol group, we instructed participants to use any assistance they would outcome. Furthermore, participants did not consistently follow these typically use at home (for example, internet search). recommendations, suggesting that the performance issues when pairing participants with LLMs may be attributed to human–LLM Task validation interaction failures. As a validation of the potential usefulness of the LLMs for addressing Second, we show that evaluations on standard benchmarks—often our specific set of ten scenarios, we provided the scenarios and ques- used to ensure safety and reliability before deployment—are not able tions directly to the models and sampled 60 responses per model per to predict human–LLM interaction failures. Medical knowledge is scenario. The models were able to suggest at least one relevant condi- typically benchmarked using questions from medical licensing exami- tion in 94.7% of cases for GPT-4o, 99.2% of cases for Llama 3 and 90.8% nations4. We compiled benchmark questions from topics matched to of cases for Command R+. The models’ accuracy in recommending our scenarios and compared LLM performance on these questions to dispositions was 64.7% for GPT-4o, 48.8% for Llama 3 and 55.5% for performance in the corresponding interactive testing for each model Command R+ (Fig. 2a). Overall, these scores indicate that the models and scenario. Performance in structured question-answering tasks, have the ability to provide useful medical information on these tasks, higher than in interactive testing as expected in 26 out of 30 instances, performing better than random guessing between the five choices of was largely uncorrelated to the interactive testing. disposition or choosing medical conditions from our gold-standard Nature Medicine | Volume 32 | February 2026 | 609–615 610 Article https://doi.org/10.1038/s41591-025-04074-y a c Online participants (n = 1,298) assigned scenario and treatment Three doctors collectively wrote ten detailed medical scenarios Specific case details You are playing the part of a 20-year-old male patient who GPT-4o Llama 3 Command R+ Control is suddenly experiencing a really severe headache. The pain (n = 340) (n = 343) (n = 314) (n = 301) developed on a Friday night while out at the cinema with friends […]. General life details My head really hurts […], what should I do? You are a university student studying for a degree in electrical engineering […]. Online It sounds like you might have meningitis […]. assistance Additional medical history without LLM You do not have any long-term health conditions, nor family … history of any health conditions […]. Okay, I’ll go to urgent care. Thanks! b d Gold standard answers combine disposition (initial three doctors) with plausible differential diagnoses (four additional doctors) Participants identified Disposition Explanation dispositions and Ambulance “I chose urgent care conditions because the A&E symptoms could Urgent Primary Care mean meningitis.” Routine GP Disposition Relevant conditions Self-care Ambulance Subarachnoid hemorrhage A&E Meningitis Urgent Primary Care Cluster headache Routine GP … Responses scored Disposition Relevant conditions using gold standard Urgent Primary Care Meningitis Self-care answers Fig. 1 | Study design. a, Three doctors drafted ten medical scenarios, iteratively scenarios. The treatment groups conversed with an LLM to help assess the revising them until they reached unanimous agreement about the best scenarios. The control group was permitted to use any method, with most disposition on a five-point scale from self-care to ambulance. b, Four additional participants using internet search or their own knowledge. d, Top: participants doctors read the scenarios and provided differential diagnoses, which were then chose a disposition and identified medical conditions that motivated their combined to form gold-standard lists of relevant conditions. c, We recruited choice. Participants completed two scenarios, until a total of 600 examples 1,298 participants and randomly assigned them to one of four experimental were collected for each experimental condition. Bottom: we evaluated each conditions. Each participant was randomly allocated one of ten medical participant’s responses using the gold-standard answers. lists to recommend at random and consistent with the strong perfor- n1 = n2 = 600; GPT-4o, F = 0.536, P = 0.023; Llama 3, F = 0.529, P = 0.072; mance of these models on other medical benchmarks6,22,27,28. Command R+, F = 0.514, P = 0.366, unadjusted). Participants using LLMs consistently performed worse than Experimental performance when the LLMs were directly provided with the scenario and task Figure 2b shows that participants using LLMs were significantly less (Fig. 2). For identifying relevant conditions, all three treatment groups likely than those in the control group to correctly identify at least performed worse than their corresponding models without human one medical condition relevant to their scenario (χ2(1), n1 = n2 = 600, interaction (χ2(1), P < 0.001, n1 = n2 = 600). In the case of disposition, P < 0.001 for all three models) and identified fewer relevant condi- GPT-4o and Command R+ performed better than any group of par- tions on average (GPT-4o, 0.42–0.54; Llama 3, 0.39–0.50; Command ticipants using LLMs (χ2(1), P < 0.001, n1 = n2 = 600), and the observed R+, 0.34–0.43; control, 0.55–0.67; bootstrap 95% confidence interval mean was higher for Llama 3 as well but not statistically significant (CI) with 1,000 resamples). Participants in the control group had 1.76 (χ2(1) = 2.44, P = 0.118, n 1 = n2 = 600). Strong performance from (95% CI = 1.45–2.13) times higher odds of identifying a relevant condi- the LLMs operating alone is not sufficient for strong performance tion than the aggregate of the participants using LLMs. They were also with users. 1.57 (95% CI = 1.28–1.92) times more likely to identify conditions from the more serious ‘red flag’ list. Performance in user interactions Participants using LLMs did not have statistically significant dif- To isolate the role of user interactions, we compared the performance ferences in disposition accuracy from the control group (GPT-4o, in identifying relevant conditions with different degrees of user involve- χ2(1) = 0.17, P = 0.683; Llama 3, χ2(1) = 0.34, P = 0.560; Command ment. In the user interactions, GPT-4o mentioned a relevant condi- R+, χ2(1) = 0.03, P = 0.861; n1 = n2 = 600; Fig. 2b). The overall correct tion in 65.7 ± 6.2% of cases, Llama 3 in 67.0 ± 6.1% and Command R+ response rate of 43.0% ± 2.0% exceeds a random guessing baseline of in 73.2 ± 5.7% (Fig. 3). Each of these was significantly lower than the 20%, but most participants still chose an incorrect disposition. Partici- performance of the LLMs alone (Fig. 2) and suggests that necessary pants using LLMs tended to underestimate the acuity of their condi- information about the scenario was not communicated between the tions, as did the control group (Mann–Whitney U n1 = n2 = 600, P < 0.001 user and the model. Despite these correct suggestions appearing in for all experimental conditions). Users of GPT-4o and Llama 3 had an the conversations, users did not consistently include them in the final observed tendency toward higher estimates of clinical acuity than the responses, indicating a second breakdown in communication between control group, but this result was not significant (Mann–Whitney U the model and user (Fig. 3). Nature Medicine | Volume 32 | February 2026 | 609–615 611 Article https://doi.org/10.1038/s41591-025-04074-y a b 100 Relevant conditions (%, 95% CI) 80 60 40 20 0 In response to direct query In user final response 100 Disposition (%, 95% CI) 80 60 40 20 0 In response to direct query In user final response GPT-4o (n = 600) Llama 3 (n = 600) Command R+ (n = 600) Control (n = 600) Fig. 2 | Performance of LLMs alone and with users. a, The performance on the The control group was significantly better than those using LLMs at identifying LLMs when directly prompted to complete each task alone. Top, the proportion relevant conditions. Differences in disposition accuracy were not statistically of LLM responses that identified relevant conditions. Bottom, the proportion of significant. Data are presented as mean values with unadjusted 95% CIs for LLM responses correctly identifying the best disposition. b, The performance proportions. Markers indicate means for each scenario. Using LLMs worsened of participants across the four experimental conditions. Top: the proportion or did not improve participant performance on these tasks relative to using of participant responses that identified relevant conditions. Bottom: the traditional resources, and the models consistently performed better without proportion of participant responses correctly identifying the best disposition. user interaction. 100 on average. We found that user final responses had only slightly bet- ter precision, 38.7% (36.3–41.4% 95% CI), than the combination of all 80 the intermediate conditions mentioned by LLMs. This indicates that Relevant conditions (%) participants may not be able to identify the best conditions suggested by LLMs. 60 To better understand the mechanisms that lead to lower perfor- mance in human–LLM interactions, we analyzed a random selection 40 of 30 interactions, one for each combination of model and scenario. For each selected interaction, we read the interaction transcript and 20 recorded (1) whether the user had provided sufficient information to correctly identify the condition in their initial message, (2) whether 0 the user had provided sufficient information over the course of the Mentioned in user interaction interaction, (3) the accuracy of any suggestions made by the model GPT-4o (n = 600) Llama 3 (n = 600) Command R+ (n = 600) and (4) whether the user ultimately followed the recommendation Fig. 3 | Identification of relevant conditions in interaction. The proportion of the model. We also noted interactions that demonstrated unique of responses where at least one condition from the gold-standard list dynamics between LLM and user. (Supplementary Table 11) was mentioned during the interaction between the Overall, users often failed to provide the models with sufficient participants and LLMs. Data are presented as mean values with unadjusted 95% information to reach a correct recommendation. In 16 of 30 sampled CIs for proportions. Markers indicate means for each scenario. interactions, initial messages contained only partial information (see Extended Data Table 1 for a transcript example). In 7 of these 16 interac- tions, users mentioned additional symptoms later, either in response To measure whether participants received accurate suggestions to a question from the model or independently. from LLMs, we searched each conversation to identify which medical LLMs generated several types of misleading and incorrect informa- conditions were mentioned (‘Condition extraction’ in Methods). We tion. In two cases, LLMs provided initially correct responses but added found that, on average, LLMs suggested 2.21 (2.12–2.32 95% CI) pos- new and incorrect responses after the users added additional details. In sible conditions per interaction, of which only 34.0% (32.3–35.9% 95% two other cases, LLMs did not provide a broad response but narrowly CI) were correct. After their interactions with LLMs, participants were expanded on a single term within the user’s message (‘pre-eclampsia‘ asked to list all relevant conditions and listed 1.33 (1.28–1.38 95% CI) and ‘Saudi Arabia’) that was not central to the scenario. LLMs also made Nature Medicine | Volume 32 | February 2026 | 609–615 612 Article https://doi.org/10.1038/s41591-025-04074-y a Passing score Passing score 100 100 Experimental disposition conditions accuracy (%) Experimental relevant 80 80 accuracy (%) 60 60 40 40 20 20 0 0 0 20 40 60 80 100 0 20 40 60 80 100 Question-answering benchmark accuracy (%) Question-answering benchmark accuracy (%) b 100 100 80 80 accuracy (%) accuracy (%) Human user Human user 60 60 40 40 20 20 0 0 0 20 40 60 80 100 0 20 40 60 80 100 Simulated user accuracy (%) Simulated user accuracy (%) Disposition Relevant condition Command R+ GPT-4o Llama 3 Fig. 4 | Model baselines. a, The accuracy of each model in responding to interactions in 26 out of 30 cases for the dispositions and all 30 cases for relevant questions from MedQA relevant to each scenario, as compared with the conditions. b, The accuracy of simulated users in identifying the best disposition performance of the human participants using the same model in the main and relevant conditions as compared to human users in the main study. Dashed study. Data are presented as mean values for each scenario and model. The lines show ordinary least squares regressions with shaded 95% CIs of the human passing standard for MedQA is 60%, which the LLMs mostly achieved. regression coefficients. Scores on question-answering are higher than the corresponding scores in user errors in contextual understanding by, for example, recommending accuracy on MedQA than users did when using the LLMs in the main calling a partial US phone number and, in the same interaction, recom- study (GPT-4o, 20 out of 20 cases; Llama 3, 19 out of 20 cases; Command mending calling ‘Triple Zero’, the Australian emergency number. Com- R+, 17 out of 20 cases). The approximate standard for a passing score paring across scenarios, we also noticed inconsistency in how LLMs is 60%, which the models typically achieved4. However, benchmark responded to semantically similar inputs. In an extreme case, two users scores of more than 80% still corresponded to human experimental sent very similar messages describing symptoms of a subarachnoid scores below 20% in several cases, indicating the potential size of the hemorrhage but were given opposite advice (Extended Data Table 2). differences. Success in question-answering tasks is not a sufficient One user was told to lie down in a dark room, and the other user was indication of whether the same information will be effectively applied given the correct recommendation to seek emergency care. Despite to real-world tasks. all these issues, we also observed successful interactions where the user redirected the conversation away from mistakes, indicating that Simulated patient interactions non-expert users could effectively manage LLM errors in certain cases To compare with benchmarks based on interactions with simulated (Extended Data Table 3). patients, we conducted a variant of our human subject experiment Participants employed a broad range of strategies when interact- with the participants replaced by LLMs. We prompted an LLM instance ing with LLMs. Several users primarily asked closed-ended questions to act as a patient, provided it with a scenario and the two questions to (for example, ‘Could this be related to stress?’), which constrained the answer for the task and instructed it to use the assistance of another possible responses from LLMs. When asked to justify their choices, two LLM to answer the questions. The ‘patient’ LLM was instructed to begin users appeared to have made decisions by anthropomorphizing LLMs a conversation with the assistant, and then chat messages were passed and considering them human-like (for example, ‘the AI seemed pretty between the two models until the ‘patient’ model answered the two task confident’). On the other hand, one user appeared to have deliberately questions. Each scenario was repeated ten times for each experimental withheld information that they later used to test the correctness of condition, resulting in a total of 300 simulated conversations. the conditions suggested by the model. Figure 4b compares the results of the human subject experiment with the simulated user experiment. For identification of dispositions, Question-answering benchmarks the simulated participants showed less variation, with 26/30 scenarios To assess how well question-answering benchmarks predicted perfor- having either 100% or 0% accuracy across ten trials. On average, the sim- mance in user deployments, we scored the LLMs on a targeted subset of ulated participants performed better than the human participants, with the popular MedQA benchmark4. Using the physician-generated lists 57.3% ± 5.6% accuracy in determining a disposition and 60.7% ± 5.5% of relevant conditions for each scenario, we filtered for MedQA ques- accuracy in identifying relevant conditions. Despite consulting with the tions that included those conditions, resulting in a list of 236 items. We same LLMs for advice, the mean scores per scenario of the simulated then scored the LLMs for accuracy on these multiple-choice questions participants in selecting a disposition were only weakly predictive of using standard five-shot prompting20,27. the real participants, with linear regression coefficients of 0.33 ± 0.25 Figure 4 shows the accuracy of each LLM on the filtered subsets of for GPT-4o users, 0.31 ± 0.31 for Llama 3 users and 0.20 ± 0.38 for Com- the MedQA benchmark for each scenario. LLMs consistently had higher mand R+ users. The scores for identifying relevant conditions showed Nature Medicine | Volume 32 | February 2026 | 609–615 613 Article https://doi.org/10.1038/s41591-025-04074-y no relationship at all, with linear regression coefficients of −0.01 ± 0.34 once LLMs become successful in scenario-based experiments, testing for GPT-4o users, −0.17 ± 0.29 for Llama 3 users and −0.01 ± 0.51 for could proceed toward increasingly realistic conditions. Command R+ users. Based on this comparison, simulated participants In our work, we found that none of the tested language models did not seem to accurately reflect human–LLM interactions, making were ready for deployment in direct patient care. Despite strong per- it crucial to include actual humans in safety testing. formance from the LLMs alone, both on existing benchmarks and on our scenarios, medical expertise was insufficient for effective patient Discussion care. Our work can only provide a lower bound on performance: newer Our findings highlight the challenges of public deployments of LLMs models, models that make use of advanced techniques from chain of for direct patient care. We have conducted a randomized study testing thought to reasoning tokens, or fine-tuned specialized models, are the effects of using an LLM to support medical self-assessment. Despite likely to provide higher performance on medical benchmarks. It is LLMs alone having high proficiency in the task, the combination of LLMs unclear, however, whether these gains will translate into higher per- and human users was no better than the control group in assessing formance with real users or only emphasize the gap when operating clinical acuity and worse at identifying relevant conditions. Previous with them. We recommend that developers, as well as policymakers work has shown that using LLMs does not improve clinical reasoning and regulators, consider human user testing as a foundation for bet- in physicians29, and we found that this extends to the general public as ter evaluating interactive capabilities before any future deployments. well. We further identified the transmission of information between the LLM and the user as a particular point of failure, with both users Online content providing LLMs with incomplete information and LLMs suggesting Any methods, additional references, Nature Portfolio reporting sum- correct answers but not effectively conveying this information to the maries, source data, extended data, supplementary information, users. We considered two common testing approaches for medical acknowledgements, peer review information; details of author contri- capabilities in LLMs and found that although they may assess the medi- butions and competing interests; and statements of data and code avail- cal information stored in the LLMs, they do not reflect the challenges ability are available at https://doi.org/10.1038/s41591-025-04074-y. of user interactions in deployment. We highlighted three aspects of user interaction identified in References our study to motivate further research. First, over the course of the 1. O’Neill, M. ChatGPT diagnoses cause of child’s chronic pain after interactions, LLMs typically offered 2.21 possible options, giving users 17 doctors failed. Independent (13 September 2023). the final decision of which to accept, but users performed poorly at 2. Presiado, M., Montero, A., Lopes, L. & Hamel, L. KFF Health making this choice. Because we showed that LLMs alone perform the Misinformation Tracking Poll: Artificial Intelligence and Health task better than most users, improvements in communicating infor- Information (KFF, 2024). mation from LLMs to users would be highly impactful. Interactive, 3. Shahsavar, Y. & Choudhury, A. User intentions to use ChatGPT multiturn evaluations like this study are key for better understanding for self-diagnosis and health-related purposes: cross-sectional and improving these capabilities30. Second, as with a real doctor– survey study. JMIR Hum. Factors 10, e47564 (2023). patient interaction, in this study the users choose what to tell the LLMs, 4. Jin, D. et al. What disease does this patient have? A large-scale which led to cases where LLMs were not given enough information to open domain question answering dataset from medical exams. provide correct advice. In clinical practice, doctors conduct patient Appl. Sci. 11, 6421 (2021). interviews to collect the key information because patients may not 5. Ayers, J. W. et al. Comparing physician and artificial intelligence know what symptoms are important, and similar skills will be required chatbot responses to patient questions posted to a public social for patient-facing AI systems. Third, the sensitivity of LLMs to small media forum. JAMA Intern. Med. 183, 589–596 (2023). variations in inputs creates challenges for forming mental models of 6. Van Veen, D. et al. Adapted large language models can LLM behavior31. Even occasional factual and contextual errors could outperform medical experts in clinical text summarization. Nat. lead users to disregard advice from LLMs32. For a public-facing medi- Med. 30, 1134–1142 (2024). cal LLM to exist, we expect that LLMs will first need to be made more 7. Agarwal, N., Moehring, A., Rajpurkar, P. & Salz, T. Combining consistent to improve user-LLM interaction. Human Expertise with Artificial Intelligence: Experimental Evidence With millions of people consulting LLMs for medical advice from Radiology. Working paper 31422 (NBER, 2023). regularly2,3, healthcare practitioners now need to know what to expect 8. Brodeur, P. G. et al. Superhuman performance of a large language from patients with LLM-based opinions about their care. We found that model on the reasoning tasks of a physician. Preprint at patients using LLMs have low accuracy in understanding the acuity of https://arxiv.org/abs/2412.10849 (2024). their symptoms and in identifying the etiology, comparable to partici- 9. Rajpurkar, P. & Topol, E. J. The robot doctor will see you now. pants using traditional approaches. Our scenarios focused on common New York Times (2 February 2025). conditions where users may be familiar with the symptoms, and results 10. Dietvorst, B. J., Simmons, J. P. & Massey, C. Algorithm aversion: might differ on rare conditions or less typical presentations. Develop- people erroneously avoid algorithms after seeing them err. J. Exp. ers of general-purpose LLM platforms may have an incentive to design Psychol. Gen. 144, 114–126 (2015). risk-adverse LLMs that are more likely to suggest consulting doctors 11. Babic, B., Gerke, S., Evgeniou, T. & Cohen, I. G. Beware or visiting emergency services. In this study, we found no significant explanations from AI in health care. Science 373, 284–286 evidence that participants who consulted LLMs had higher estimates (2021). of the acuity of their scenarios, with only a small difference observed. 12. Doshi, A. R. & Hauser, O. P. Generative artificial intelligence This should, however, be closely monitored in the future to ensure that enhances creativity but reduces the diversity of novel content. health services are not overwhelmed with spurious requests as LLM Sci. Adv. 10, eadn5290 (2024). usage increases and diversifies. 13. Digital access – a ‘front door to the NHS’. NHS England South East In line with recent work on the use of LLMs in medicine, we used https://www.england.nhs.uk/south-east/info-professionals/ clinical vignettes for the interactions to limit risks to participants. south-east-region-primary-care-transformation-programme/ This allowed us to focus on the challenges of LLMs interacting with introduction/digital-access-a-front-door-to-the-nhs/ (2025). members of the general public, without taking into account partici- 14. Kyle, P. AI opportunities action plan. GOV.UK www.gov.uk/ pants’ feelings of urgency and stress to make a good decision based on government/publications/ai-opportunities-action-plan/ life-threatening symptoms they are experiencing. We recommend that ai-opportunities-action-plan (2025). Nature Medicine | Volume 32 | February 2026 | 609–615 614 Article https://doi.org/10.1038/s41591-025-04074-y 15. Catalyzing equitable artificial intelligence (AI) use to improve 27. Saab, K. et al. Capabilities of Gemini models in medicine. Preprint global health. Global Grand Challenges https://gcgh. at https://arxiv.org/abs/2404.18416 (2024). grandchallenges.org/challenge/catalyzing-equitable-artificial- 28. Tu, T. et al. Towards conversational diagnostic artificial intelligence-ai-use (2023). intelligence. Nature 642, 442–450 (2024). 16. US Department of Health and Human Services. HHS releases 29. Goh, E. et al. Large language model influence on diagnostic strategic plan for the use of artificial intelligence to enhance and reasoning: a randomized clinical trial. JAMA Netw. Open 7, protect the health and well-being of Americans. https://www.hhs. e2440969 (2024). gov/about/news/2025/01/10/hhs-releases-strategic-plan-use- 30. Ibrahim, L. et al. Towards interactive evaluations for interaction artificial-intelligence-enhance-protect-health-well-being- harms in human-AI systems. In Proc. Eighth AAAI/ACM Conference americans.html (2025). on AI, Ethics, and Society 1302–1310 (AAAI, 2025). 17. Haupt, C. E. & Marks, M. AI-generated medical advice—GPT and 31. Wilder, B., Horvitz, E. & Kamar, E. Learning to complement beyond. JAMA 329, 1349–1350 (2023). humans. In Proc. 29th International Joint Conference on Artificial 18. Lee, P., Bubeck, S. & Petro, J. Benefits, limits, and risks of GPT-4 Intelligence (ed. Bessiere, C.) 1526–1533 (ACM, 2020). as an AI chatbot for medicine. N. Engl. J. Med. 388, 1233–1239 32. Jones-Jang, S. M. & Park, Y. J. How do people react to AI (2023). failure? Automation bias, algorithmic aversion, and perceived 19. Moor, M. et al. Foundation models for generalist medical artificial controllability. J. Comput. Mediate. Commun. 28, zmac029 intelligence. Nature 616, 259–265 (2023). (2023). 20. Singhal, K. et al. Large language models encode clinical knowledge. Nature 620, 172–180 (2023). Publisher’s note Springer Nature remains neutral with regard 21. Tu, T. et al. Towards generalist biomedical AI. NEJM AI to jurisdictional claims in published maps and institutional https://doi.org/10.1056/AIoa2300138 (2024). affiliations. 22. Nori, H., King, N., McKinney, S. M., Carignan, D. & Horvitz, E. Capabilities of GPT-4 on medical challenge problems. Preprint at Open Access This article is licensed under a Creative Commons https://arxiv.org/abs/2303.13375 (2023). Attribution 4.0 International License, which permits use, sharing, 23. Schmidgall, S. et al. AgentClinic: A multimodal agent benchmark adaptation, distribution and reproduction in any medium or format, to evaluate AI in simulated clinical environments. Preprint at as long as you give appropriate credit to the original author(s) and the https://arxiv.org/abs/2405.07960 (2024). source, provide a link to the Creative Commons licence, and indicate 24. Johri, S. et al. An evaluation framework for clinical use of large if changes were made. The images or other third party material in this language models in patient interaction tasks. Nat. Med. 31, 77–86 article are included in the article’s Creative Commons licence, unless (2025). indicated otherwise in a credit line to the material. If material is not 25. Reis, M., Reis, F. & Kunde, W. Influence of believed AI involvement included in the article’s Creative Commons licence and your intended on the perception of digital medical advice. Nat. Med. 30, use is not permitted by statutory regulation or exceeds the permitted 3098–3100 (2024). use, you will need to obtain permission directly from the copyright 26. Milanovic, K. & Pitt, J. Misattribution of error origination: the holder. To view a copy of this licence, visit http://creativecommons. impact of preconceived expectations in co-operative online org/licenses/by/4.0/. games. In Proc. ACM Designing Interactive Systems Conference (eds Ju, W. et al.) 707–717 (ACM, 2021). © The Author(s) 2026, modified publication 2026 Nature Medicine | Volume 32 | February 2026 | 609–615 615 Article https://doi.org/10.1038/s41591-025-04074-y Methods level of education, English fluency, internet usage habits, medical The study employed a between-subjects design with three treatment expertise and experience using LLMs. Tests of confounding effects of groups and a control. Before the treatment phase, participants com- these variables are included in Supplementary Tables 5–8. pleted a demographic survey. In the treatment phase, we presented study participants with medical scenarios that could arise in daily life Treatments and instructed them to assess the clinical acuity. Following the practice Participants were assigned to one of three treatment groups or a con- of recent work testing medical LLMs24,28,29, we used clinical vignettes to trol. The treatment groups were provided with an LLM chat interface enable comparable conditions across the treatment groups. In treat- to use in support of completing the task. To represent different types ment groups, we provided participants with an LLM chat interface to of models that might be used by a member of the public, we selected support their decision-making, and we instructed the control group three different leading LLMs: GPT-4o, chosen for its large user base to use any other reference material they would ordinarily use. After as the model most likely to be used by the general public; Llama 3, completing the treatment, all participants completed a post-survey selected for its open weights, most likely to be used as the backbone about their experience. Data collection was conducted using the Dyna- for creating specialized medical models; and Command R+, included bench platform33, with pre- and post-treatment surveys conducted via for its use of retrieval-augmented generation, a process in which the Qualtrics. We conducted three small pilot studies with the same target model searches the open internet for information before generating population to refine the interface and instructions, resulting in changes responses, potentially increasing reliability. The control group par- to the instruction texts and a shift from presenting the scenarios in the ticipants were instructed to use any source they would typically use at third person to the second person. We preregistered our study design, home. Post-treatment surveys indicated that most participants used a data collection strategy and analysis plan for the human subject experi- search engine or went directly to trusted websites, most often the NHS ment (https://osf.io/dt2p3). The LLM-only experiments were added after website (Extended Data Fig. 1 and Extended Data Table 4.) Participants preregistration to better understand the results of the human study. were blinded to which model they had been assigned and would not be able to distinguish based on the interface. The control group were Participants necessarily aware that they were not using an LLM. The models were We used Prolific to recruit 1,298 participants to collect 2,400 conver- queried via API endpoints from OpenAI, Hugging Face and Cohere, sations. All participants were required to be over the age of 18 and respectively. The hyperparameters and inference costs are listed in speak English. Data collection was run between 21 August 2024 and Supplementary Tables 9 and 10. 14 October 2024. We used stratified random sampling via the Prolific platform to target a representative sample of the UK population in each Scenarios group. The sample size was chosen based on the minimum number of We assigned each participant two scenarios to complete consecutively. participants required to provide sufficient demographic coverage for The scenarios describe patients who are experiencing a health condition this sample. The participants were paid £2.25 each. All data collection in everyday life and need to decide whether and how to engage with the was pseudonymous, and unique pseudonyms were disassociated healthcare system. We focused on medical scenarios encountered in from the data before publication. Informed consent was obtained everyday life because this is a realistic setting for the use of LLMs when from all participants. For a power analysis and detailed demograph- professional medical advice is not at hand. For robustness, we created ics of the participants, as well as a breakdown of the results by sex, see ten medical scenarios spanning a range of conditions with different pres- Supplementary Information. entations and acuities, which were assigned to participants at random. During data collection, we encountered an API issue where the To assess the decisions being made, we asked participants two LLMs failed to provide a response before timing out; the issue cascaded questions about the scenarios: (1) “What healthcare service do you on our platform to impact the GPT-4o and Llama 3 treatment groups, need?” and (2) “Why did you make the choice you did? Please name all requiring 98 participants to be replaced because the models failed to specific medical conditions you consider relevant to your decision”. respond to the users. We paid all of the impacted participants. We also These questions captured the accuracy of decisions being made as replaced 13 participants who appeared in more than one treatment well as the understanding participants developed in reaching their group due to a software error in the Prolific platform. Due to the techni- decision. The questions were available to participants at all times dur- cal issues, we adapted our stopping protocol from the original prereg- ing this phase of the study. istration to collect 600 interactions per treatment with a maximum of The first question was multiple choice, with the five options shown two per participant instead of having exactly 300 participants per treat- in Extended Data Table 5, mapping acuity levels onto the UK healthcare ment with 2 interactions each. This resulted in variations in the number system. For clarity, participants were provided with the options as of participants in each treatment group. In addition to replacements well as their descriptions, although many participants were already for technical issues, we excluded data from 493 participants who began generally familiar with their local healthcare systems. The most acute, but did not complete the study. Of these participants, 392 began only ambulance, is an appropriate response to conditions where treatment the presurvey and were not exposed to a treatment. For the remaining is needed in transport before the patient arrives at a hospital. The sec- 101, we found no evidence of an association between attrition rate and ond most acute, accident and emergency, is the appropriate response treatment group, with 26 dropping out from the GPT-4o treatment, 30 to conditions where hospital treatment is needed. The middle option, from the Llama 3 treatment, 25 from the Command R+ treatment and urgent primary care, is the appropriate response to conditions where 20 from the control group (χ2(3) = 0.948, d.f. = 3, P = 0.814). the patient needs to be seen urgently by a medical professional but does The study protocols followed in this study were approved by the not need specialized hospital treatment. The second least acute option, Departmental Research Ethics Committee in the Oxford Internet Insti- routine general practitioner, is the appropriate response to conditions tute (University of Oxford) under project number OII_C1A_23_096. that require expert medical advice but are not urgent in nature. The Methods were carried out in accordance with the relevant guidelines least acute option, self-care, is the appropriate response for conditions and regulations. Informed consent was obtained from all participants that can be managed by the patient without expert medical support. before enrollment in the study. The second question was free response. We provided an example, “suspected broken bone”, to encourage participants to be specific in Presurvey their answers. We also asked participants how confident they were Before the interaction phase, we collected information from partici- in their response using a visual analog scale, as in previous studies pants about their backgrounds. We asked participants to report their measuring confidence in judgments34. Nature Medicine Article https://doi.org/10.1038/s41591-025-04074-y We created ten new medical scenarios under the direction of three than the control, and to assess the tendency of the participants to physicians and established gold-standard responses based on assess- over- or underestimate the acuity of their conditions, to include infor- ments by four additional physicians. We worked with five general prac- mation about the degree of errors, with the common language effect U titioners and two intensive care doctors with an average of 24 years of size, f = n1 n2 , used to report the effect size. experience practicing medicine. We initially drafted ten scenarios using For the CIs of the number of conditions appearing within the con- ‘Clinical Knowledge Summaries’ from the UK National Institute for versations, we used bootstrapping, taking 1,800 samples with replace- Health and Care Excellence (NICE) guidelines35. We selected common ment to compute the mean number of conditions and repeating this conditions with symptoms that could be described without special- 1,000 times. ized medical terminology. The first three physicians then revised each For the comparisons to the simulated baseline, linear regressions scenario iteratively until they agreed that the best disposition, based and CIs were computed with the SEABORN v0.13.2 regression plot on our five-point scale, was unambiguous to a trained professional. function as well as STATSMODELS. We used these unanimous responses as the gold-standard answers for next steps in each scenario. Post-survey The four other physicians then reviewed the scenarios without Following the interactive phase, we asked patients to provide com- knowing the intended responses and provided a list of differential mentary on their experience. For each interaction, participants rated diagnoses and ‘red flag’ conditions for each scenario. In every case, the their reliance on different sources of information when making their physicians’ differentials included the conditions we had used as the decisions. They also provided ratings of trust in LLMs for general and basis of the scenarios, indicating the clarity of the scenario texts. We medical purposes and the likelihoods that they would recommend took the union of the differential conditions listed by these doctors to LLMs to their family and friends. Results of the post-survey are included create a gold-standard list of relevant and red-flag conditions for each in Extended Data Figure 1. scenario. By creating new scenarios, we ensured that there was no direct overlap with the LLMs’ training data, meaning any correct responses Condition extraction would require generalizations of other information. To identify how many conditions were mentioned by the LLMs in each Each scenario follows a consistent three-part format, with Specific interaction, we used GPT-4o to extract a list of conditions. We used the Case Details giving the presenting symptoms within a short history; prompt “Identify and return the names of any medical conditions listed General Life Details giving a brief description of the patient’s lifestyle; in the **Response**. If there is more than one condition present, return and Additional Medical History giving long-term conditions, smoking, them all in a comma-separated list. If there are no conditions present, drinking and dietary habits. Each scenario includes additional informa- then return ‘None’. If there is a main condition with a concurrent minor tion beyond what is necessary for the case at hand so that participants condition, return the name of the main condition. Do not explain”. are forced to identify the most relevant information, as they would in a For each interaction, this produced a list of different conditions that real situation. Scenarios are written in the second person (for example, appeared. We applied the same approach to the user final responses ‘You have a persistent headache…’) based on common practice36,37. to produce a count of conditions mentioned there as well. The specific cases are described in brief in Extended Data Table 6, and the full texts and gold-standard relevant conditions are included Direct model prompting in Supplementary Information. To test the medical assessment abilities of the models independent of the users, we prompted each model with the text of the scenarios and Scoring asked directly for the answers to the same questions provided to the We scored the question responses for accuracy in assessing the clini- users. We prompted the model separately for disposition and relevant cal acuity. We considered responses to be correct if they matched the conditions to avoid enforcing an answer format that could impact the expected disposition for the scenario as determined by the physicians. quality of the responses. In initial testing, the models were less likely to We reported separately the rates of over- and underestimation of sever- follow the required multiple-choice format when both questions were ity, because the consequences of each are different. This accuracy is asked together. For consistency with the rest of the study, we used the a key metric for the success of the human–LLM teams, because in real same hyperparameters and zero-shot prompting and collected 600 cases, subsequent care would depend on reaching the right provider. responses from each model. We scored the free-text explanations based on whether the relevant conditions named were consistent with those in our Question-answering baseline physician-generated gold-standard list. Because we asked participants To compare with the performance of these models on existing bench- to list ‘all’ potentially relevant conditions, rather than penalizing wrong marks, we scored each model on the subset of MedQA4 that was most answers, an answer was counted as correct if one of the mentioned con- relevant to our scenarios. MedQA is the most widely used medical ditions matched a gold-standard condition. Participants listed an aver- question-answering benchmark and uses questions from medical age of 1.33 conditions, making this metric similar to requiring a single licensing exams. To select relevant subsets, we filtered for questions answer to exactly match the gold-standard list. We used fuzzy matching that contained conditions in our gold-standard lists of relevant con- to allow for misspellings and alternate wordings of the same condition. ditions. We then scored the models on only those questions using We used a manually scored sample of 200 cases to choose the threshold five-shot prompting, where models are given five examples of the that matched as many correct responses as possible without accepting question/answer format before the real question20. We tested on 236 incorrect responses. We chose a threshold of 20% character difference, questions from MedQA4 that mentioned conditions relevant to each which had a precision of 95.8% and a recall of 95.8%, allowing two false scenario: subarachnoid hemorrhage (n = 10), pulmonary embolism negatives and two false positives (Supplementary Fig. 6). (n = 30), tinnitus (n = 6), ulcerative colitis (n = 19), renal colic (n = 11), gallstones (n = 26), pneumonia (n = 40), anemia (n = 40), common cold Statistical methods (n = 24) and allergic rhinitis (n = 30). Details of the question-filtering All statistics were computed using the STATSMODELS v0.14.3 and SCIPY procedure are in Supplementary Information. v1.13.0 packages in Python. Comparisons between proportions were computed using χ2 tests with 1 d.f., equivalent to a two-sided Z-test. Simulated participants baseline Two-sided Mann–Whitney U tests were used to test the probability of To compare with the performance of these models on tasks with simu- responses from each treatment group rating the acuity more highly lated users, we replicated the human study experimental design with Nature Medicine Article https://doi.org/10.1038/s41591-025-04074-y LLM-simulated participants adapted from previous works. For the Acknowledgements simulated participants, we used the following prompt based on the A.M. acknowledges support from Prolific and support for the equivalent prompt in ref. 24, which is the state of the art for simulated Dynabench platform from the Data-centric Machine Learning Working medical interactions: “You are a patient. You do not have any medical Group at MLCommons. A.M. and A.M.B. were partially supported by knowledge. You have to self-assess your symptoms from the given case the Oxford Internet Institute’s Research Programme funded by the vignette and assistance from an AI model. Do not break character and Dieter Schwarz Stiftung gGmbH. L.R. acknowledges support from the reveal that you are describing symptoms from the case vignette. Do Royal Society Research grant no. RG\R2\232035 and the UKRI Future not generate any new symptoms or knowledge, otherwise you will be Leaders Fellowship (grant no. MR/Y015711/1). L.T. was supported by penalized. Remember, you are the patient. Simplify terminology used the National Institute for Health Research (NIHR) Oxford Biomedical in the given paragraph to layman language and keep your questions or Research Centre (BRC). The views expressed are those of the authors statements reasonably short”. and not necessarily those of the NHS, the NIHR or the Department of As in other works, we used GPT-4o for all of the simulated Health. The funders had no role in study design, data collection and participants23,24. This ensured a fair comparison between the models, analysis, decision to publish or preparation of the manuscript. as the patients would be identical. Author contributions Reporting summary A.M.B., L.R. and A.M. were involved in the conceptualization of the Further information on research design is available in the Nature study. A.M.B., R.P., G.P., H.R.K., L.R. and A.M. contributed to the Portfolio Reporting Summary linked to this article. methodology. R.E.P., G.P. and A.S.E. provided medical expertise and created the scenarios. A.M.B., J.C., R.M.-G. and S.H.M. contributed Data availability code for the data collection and analysis. A.M.B. conducted the formal The datasets generated by the experimental research during the cur- analysis with validation and supervision from L.R. and A.M. A.M.B. rent study are available at https://github.com/am-bean/HELPMed as wrote the initial draft, with input from R.E.P., H.R.K., L.T., L.R. and A.M. well as https://huggingface.co/datasets/ambean/HELPMed/. The full Visualizations were created by A.M.B., H.R.K. and L.R. All authors text of the scenarios is available in Supplementary Information and at reviewed and approved the content of the study and are personally https://huggingface.co/datasets/ambean/HELPMed/viewer/default/ responsible for their contributions as stated here. scenarios. Competing interests Code availability The authors declare no competing interests. All code used to generate the analysis in the manuscript is shared by the authors for reuse and is available at https://github.com/am-bean/ Additional information HELPMed. Extended data is available for this paper at https://doi.org/10.1038/s41591-025-04074-y. References 33. Kiela, D. et al. Dynabench: rethinking benchmarking in NLP. In Supplementary information The online version Proc. Conference of the North American Chapter of the Association contains supplementary material available at for Computational Linguistics: Human Language Technologies https://doi.org/10.1038/s41591-025-04074-y. (eds Toutanova, K. et al.) 4110–4124 (ACL, 2021). 34. Rahnev, D. et al. The confidence database. Nat. Hum. Behav. 4, Correspondence and requests for materials should be addressed to 317–325 (2020). Adam Mahdi. 35. NICE guidance. National Institute for Health and Care Excellence. https://www.nice.org.uk/guidance/ Peer review information Nature Medicine thanks Avishek Choudhury 36. Vollmar, H. C., Ostermann, T. & Redaèlli, M. Using the scenario and the other, anonymous, reviewer(s) for their contribution to the method in the context of health and health care – a scoping peer review of this work. Primary Handling Editor: Lorenzo Righetto, in review. BMC Med. Res. Methodol. 15, 89 (2015). collaboration with the Nature Medicine team. 37. Xiong, K. Z., Shah, S., Stone, J. A., Jacobson, N. & Chui, M. A. Using a scenario-based hybrid approach to understand participant Reprints and permissions information is available at health behavior. Res. Soc. Adm. Pharm. 17, 2070 (2021). www.nature.com/reprints. Nature Medicine Article https://doi.org/10.1038/s41591-025-04074-y Extended Data Fig. 1 | Post-survey. (Upper Left) Self-reported confidence use. Participants in the GPT-4o group were more likely than those in the Control in urgency assessments, conditioned on the subsequent accuracy of the group to recommend using LLMs for medical purposes (Two-sided t-test with assessment. Box plots present median values (solid line), mean values (dashed Bonferroni adjustment for 3 comparisons, p < 0.0001) or general use (p < line), boxes corresponding to the Q1-Q3 quartiles and whiskers corresponding 0.0001). The other groups showed no significant difference to the control (Llama to Q1/Q3 ± 1.5IQR. Confidence tends to be higher when responses are correct 3, p = 0.662, p = 0.527; Command R+, p = 0.333, p = 0.370). Data are presented (Two-sided t-test, p < 0.0001). In aggregate, mean confidence is higher for the as violin plots with means. (Lower) Participants reported the degree to which Control group than the Command R+ group (Two-sided t-test with Bonferroni they relied on each potential source of information. Cells show mean next step adjustment for 3 comparisons, p = 0.002), but similar to Llama 3 80B (p = 0.129) accuracy and condition identification accuracy for users reporting each degree and GPT-4o (p = 0.754). (Upper Right) Participants ratings as to whether they of influence by source, with counts per cell in parenthesis. would recommend LLMs to friends or family for medical purposes or general Nature Medicine Article https://doi.org/10.1038/s41591-025-04074-y Extended Data Table 1 | Interaction Transcript: Incomplete Information The transcript of a participant-model interaction from the dataset. In this case, the model is Command R+ and the scenario is ‘gallstones’. Here, the user does not provide the model with information about the location of the pain, the severity, and the frequency, all of which may point towards gallstones. Nature Medicine Article https://doi.org/10.1038/s41591-025-04074-y Extended Data Table 2 | Interaction Transcript: Inconsistent Responses The transcripts of two participant-model interactions from the dataset. In both of these converstaions, the model is GPT-4o and the scenario is ‘subarachnoid haemorrhage’. Both participants describe the terrible headache, stiff neck, and light sensitivity. Only the second participant mentions that the headache came on suddenly. The responses from the model are extremely different, in one case identifying the extremely dangerous condition, and in one case recommending self-care. Nature Medicine Article https://doi.org/10.1038/s41591-025-04074-y Extended Data Table 3 | Interaction Transcript: Recovered Interaction The transcript of a participant-model interaction from the dataset. In this case, the model is Llama 3 and the scenario is ‘renal colic’. Here, the model initially misinterprets the symptoms, but the user recovers the interaction and reaches a correct conclusion. Nature Medicine Article https://doi.org/10.1038/s41591-025-04074-y Extended Data Table 4 | Post-survey Sources of Information The self-reported degree to which participants relied on each source of information by treatment group. Relative to the treatment groups, the Control group was less likely to rely on an LLM (Two-sided Mann-Whitney U n1 = n2 = 600, GPT-4o: f = . 755, p < 0.0001; Llama 3: F = . 715, p < 0.0001; Command R+: F = . 287, p < 0.0001) and more likely to rely on search (Two-sided Mann-Whitney U n1 = n2 = 600, GPT-4o: F = . 297, p < 0.0001; Llama 3: F = . 281, p < 0.0001; Command R+: F = . 726, p < 0.0001), providing evidence of compliance with the treatments. Including the limited non-compliance, interpreting the results based on the intention to treat reflects the effect of instructing participants to use LLMs. AI search overviews were in limited use at the time of data collection, but optional free-text comments from the control group indicated that the NHS website was often the ultimate source of information for participants using a search engine. Nature Medicine Article https://doi.org/10.1038/s41591-025-04074-y Extended Data Table 5 | Healthcare system entry points Participants are asked to identify the correct response to each scenario by selecting the best next step on a five-point scale modelled after entry points to the NHS. Explanations are provided to avoid differences in interpretation or idiosyncratic practical considerations. Nature Medicine Article https://doi.org/10.1038/s41591-025-04074-y Extended Data Table 6 | Medical scenarios A list of the medical scenarios used in the study. Each scenario was drafted and reviewed by a team of three practicing physicians, and proposed differential diagnoses were collected from an additional four physicians with expertise in general practice. As these are excerpts, the correct assessment may not be clear without additional context. For the complete scenario texts, see the Supplementary Materials. Nature Medicine