Die Anatomie eines persönlichen Gesundheitsagenten

Article The anatomy of a personal health agent https://doi.org/10.1038/s41586-026-11161-8 A. Ali Heydari1,5 ,RKen Gu1,5, Vidya Srinivas1,5, Hong Yu1,5, Akshay Paruchuri1,5, Zhihan Zhang1, Yuwei Zhang1, Qian He1, Hamid Palangi1, Nova Hammerquist1, Ahmed A. Metwally1, Received: 22 October 2025 Brent Winslow1, Yubin Kim2, Kumar Ayush1, Yuzhe Yang1, Girish Narayanswamy1, Accepted: 11 September 2026 Maxwell A. Xu1, Jake Garrison1, Samuel Schmidgall1, Amy Armento Lee1, Jenny Vafeiadou1, Ben Graef3, Isaac R. Galatzer-Levy2, Erik Schenck1, Andrew Barakat1, Javier Perez1, Published online: xx xx xxxx Jacqueline Shreibati1, John Hernandez1, Anthony Z. Faranesh1, Naghmeh Rezaei1, Open access Javier L. Prieto1, Conor Heneghan1, Jiening Zhan1, Mark Malhotra1, Shwetak Patel1, Yun Liu1, Tim Althoff1, Xin Liu1,6 ,RDaniel McDuff1,6 R& Xuhai Orson Xu1,4,6 R Check for updates Large language models are enabling a new generation of health agents, yet their ability to address the diverse needs of individuals in everyday health settings remains underexplored1,2. Here we developed a comprehensive personal health agent that reasons over multimodal data from consumer wellness devices and medical records to provide personalized health support. Through analysis of web search and health forum queries, together with user-centred research involving end users and health experts, we identified four key user journeys: general health knowledge, personal data interpretation, wellness advice, and symptom assessment. These journeys informed three specialist sub-agents: (1) a data science agent that analyses longitudinal wearable and medical data, incorporating population-level statistics to contextualize numerical insights; (2) a health domain expert agent that integrates personal and contextual data with health knowledge to generate personalized insights; and (3) a health coach agent that synthesizes insights, supports multi-turn interaction and goal setting, applies specified psychological strategies and tracks progress. We integrated these capabilities in the personal health agent, an open-source multi-agent framework for dynamic, personalized health interactions. We evaluated the sub-agents and full system across 11 benchmark tasks using automated and human evaluation, with more than 7,000 annotations and 1,350 h of effort from health experts and end users. This represents the most comprehensive evaluation of a personal health agent thus far and provides a concrete foundation for improving access to high-quality personalized health support more broadly. Rapid advancements in large language models (LLMs) have driven Individual needs for daily health and wellness support are highly transformative changes in health. Current LLMs can support various diverse2,20,21. We used a user-centred design process spanning end-user clinical tasks, such as medical documentation3, diagnosis4, decision queries and discussions, surveys and an expert workshop to cover gen- support5 and drug discovery6,7, as well as consumer health tools, includ- eral wellness, fitness and medical topics. The analysis identified four ing virtual health assistants8, symptom checkers9, digital coaches and categories of needs: (1) general health knowledge, (2) personal-data advisors10,11, and clinical examination dialogues12. interpretation, (3) wellness advice, and (4) health symptoms (Supple- Wearables add continuous activity, sleep, heart rate variability and mentary Table 1). These categories require different capabilities. Per- other physiological signals13,14, providing longitudinal context for sonal data questions demand numerical precision and code execution; health support. Recent systems connect these data to LLMs1,2,8,15–17, coaching requires open-ended, empathetic dialogue; and knowledge or but numerical time-series reasoning remains unreliable18. Previous symptom questions require factual grounding. Our experiments con- systems either cover narrow fitness or sleep questions without deep firmed that a monolithic LLM architecture struggles to balance these numerical analysis8, or address broader numerical queries without competing objectives. We therefore adopted a modular multi-agent evaluating health expertise and interactive coaching2. A comprehensive architecture that decouples these technical requirements while pre- system must combine these capabilities while maintaining a coherent serving a cohesive user experience through orchestration and synthesis interaction19. Building on these multifaceted developments and the of individual sub-agent processes. integrated nature of personal health journeys, we envision that the Three specialists address these requirements. The data science era of intelligent personal health agents is arriving. agent (DS agent) converts personal and population-level time series 1 Google Research, Mountain View, CA, USA. 2Google DeepMind, Mountain View, CA, USA. 3Vituity, Emeryville, CA, USA. 4Columbia University, New York, NY, USA. 5These authors contributed equally: A. Ali Heydari, Ken Gu, Vidya Srinivas, Hong Yu, Akshay Paruchuri. 6These authors jointly supervised this work: Xin Liu, Daniel McDuff, Xuhai Orson Xu. Re-mail: aliheydari@google.com; xliucs@google.com; dmcduff@google.com; orson@google.com Nature | www.nature.com | 1 Article into quantitative analyses, such as estimating changes in running performance. The domain expert agent (DE agent) interprets wear- Modular multi-agent health architecture able data, health knowledge and available medical records (in this We developed the PHA as a modular system of three specialist agents, study, medical records were limited to blood test results because of each addressing a complementary user need (Fig. 1b). protocol-level data access constraints; future work could integrate The DS agent analyses personal time-series data to derive quanti- electronic health records with other modalities), for example, by tative insights. Although LLMs have been explored for data science contextualizing a biomarker value. The health coach agent (HC agent) tasks2,28–35, their capacity to interpret ambiguous, open-ended personal applies evidence-based strategies such as motivational interview- health inquiries and perform statistically sound analyses remains a key ing22 to support goal setting, identify barriers and guide behavioural gap. Our DS agent is designed to translate such queries into robust change. Because many requests span these roles, we drew inspira- analysis plans and accurate computation code. Example conversations tion from multi-agent collaboration strategies, such as voting23,24, between a user and the DS agent are shown in Extended Data Fig. 1. iteration25, debate26 and hierarchical team formation27, and built More technical details can be found in the Methods. the multi-agent personal health agent (PHA) with an orchestrator The DE agent provides evidence-based, clinical context for user data. to assign work and synthesize the outputs of the agents (Fig. 1a). Of Previous work has benchmarked the medical knowledge of LLMs36–38, note, PHA is not intended to supplant human health providers, but but the essential skill of reasoning and synthesizing across multimodal to explore the technological frontier of consumer empowerment, personal data (for example, wearables, laboratory results and user envisioning how a unified agent can provide more holistic, action- context) is underexplored. Our DE agent is equipped with a suite of able health insights. tools to integrate these disparate data streams and generate accu- We developed a holistic evaluation framework (Fig. 1b) that maps to rate, personalized interpretations. Example conversations with the the capabilities identified through the user-centred analysis. We bench- DE agent are shown in Extended Data Fig. 2. More technical details marked the DS agent on its ability to generate robust analysis plans and are in the Methods. codes, the DE agent on its capacity for factuality and evidence-based The HC agent guides users through multi-turn conversations on goal reasoning, and the HC agent on its coaching effectiveness through setting and behavioural change. Although artificial intelligence coach- user studies with both end users and experts. We then evaluated the ing is an emerging area39–41, support for open-ended dialogue across integrated PHA system through open-ended, multi-modal conversa- a wide range of wellness topics is limited. We designed our agent with tions that reflect diverse health scenarios. Across 11 benchmark tasks, a modular architecture to provide an expert-aligned, personalized the evaluation included more than 7,000 human annotations and coaching experience that effectively balances information gathering 1,350 h of effort (651 end-user hours and 699 expert hours), validat- with actionable advice. Example conversations between a user and the ing the advantage of each specialized sub-agent and demonstrating HC agent are shown in Extended Data Figs. 3 and 4, with more details the promising real-world applicability of our integrated multi-agent in the Methods. health agent. Finally, the PHA is the fully integrated system. A central orchestra- By combining a modular multi-agent architecture, a capability- tor dynamically manages the workflow, decomposing user queries, matched evaluation framework and an open-source agent implementa- assigning tasks to the appropriate specialist agents and synthesizing tion, this work establishes a reproducible foundation for developing, their outputs through an iterative reflection process to provide a single, comparing and testing PHAs for everyday health and well-being of cohesive and context-aware response to the user42. Example multi-turn individuals. conversations with PHA are shown in Extended Data Figs. 5–8, with more technical details in the Methods. Figure 1a maps CUJs to specialist agents. For example, the query User-centred analysis of health needs ‘I want to understand my sleep data last week and know how I can We first performed a systematic, user-centred analysis to define the improve my sleep’ (CUJ2 and CUJ3) recruits the DS agent to analyse landscape of personal daily health needs. We synthesized 1,370 con- wearable data, the DE agent to provide evidence-based sleep knowl- sumer health queries from search engines and health forums, health edge and the HC agent to personalize advice. Supplementary Table 1 goal surveys from 555 users, and a design workshop with 14 UX and enumerates the 26 query sub-types and their primary and supporting engineering experts (see Methods). This analysis identified four critical agents. user journeys (CUJs; Fig. 1a): To rigorously validate this framework, we designed the most com- (1) General health knowledge: users seek factual information about prehensive evaluation of a personal health agent thus far. Our eval- broad health topics, medical conditions and wellness concepts; uations were grounded in data from the large-scale Wearables for for example, ‘what are the pros and cons of HIIT?’. Metabolic Health (WEAR-ME) study (n = 1,165) to ensure real-world (2) Personal data insights: users want to understand and interpret their validity43. own data from wearables or medical records to identify trends and This dataset provides rich, multimodal data for each participant relationships; for example, ‘how did my steps impact my resting with data-quality validation (for example, filtering users with short or heart rate in the past month?’. unreliable wearing periods, and high missing data rates), including lon- (3) Wellness advice: users look for actionable and personalized recom- gitudinal wearable data, clinical blood test results and detailed health mendations to improve their well-being or achieve specific health questionnaires (see Extended Data Fig. 9). The evaluation assesses both goals; for example, ‘to improve my sleep score, what time should I the standalone performance of each sub-agent and the synergistic go to bed?’. efficacy of the integrated PHA system across 11 benchmark tasks. We (4) Personal medical symptoms: users describe their symptoms to used the Gemini 2 family (Flash/Pro) of models44 for our evaluation, explore potential causes and understand what might be happening; which has demonstrated strong performance on medical tasks8,45,46. for example, ‘my throat is itchy and I have trouble swallowing. What In this work, our focus is not to train a new LLM but to design and con- might be going on?’. struct the agentic system around a given base LLM. This multi-faceted approach involved both automated and human evaluations, drawing Supplementary Table 1 provides a detailed breakdown of these cat- on over 7,000 annotations and 1,350 h of effort from health experts egories. Because CUJs often overlap within one conversation, they and end users. The detailed justification of the choice of base LLM, motivated the production of a modular architecture spanning data evaluation protocols and a full description of the WEAR-ME dataset analysis, health-domain reasoning and coaching. are provided in the Methods. 2 | Nature | www.nature.com a CUJs 1,370 Consumer queries synthesized from Search, Gemini, Fitbit forum CUJ1: general health knowledge CUJ2: personal data insights DS agent Example: what are the pros and cons of HIIT? Example: how did my steps impact my resting How long is strep contagious? heart rate in the past month? Fitbit Labs chatbot launch and Supporting agents: Supporting agents: experience surveys from 555 users + DE agent CUJ3: wellness advice CUJ4: personal medical symptoms Example: where can I improve my sleep score? Example: my throat is itchy and I have trouble Design workshop with 14 experts At what time should I go to bed? swallowing. What might be going on? from UX, engineering and research Supporting agents: Supporting agents: HC agent b It looks like your sleep duration is okay, but your deep sleep percentage is around Memory update 12.7%, which is below the recommended 20–25%. Also, your readiness score is averaging 48.9, which is quite a bit lower than the ideal 70–100. Identify main and What is my biggest supporting agents These two things are probably related. Collaboration Self-reflection Orchestrator problem in sleep? How can According to the National Sleep I improve? Foundation, low deep sleep can often + impact your readiness score. Before we dive into specific Personal health data recommendations, can you tell me a bit more about your current sleep habits? For example, what time do you usually go to bed and wake up? Do you have a Orchestration stage Reasoning and response stage bedtime routine? Do you feel rested when you wake up? PHA DS agent DE agent HC agent Data Clinical Data availability Personalized transformation consistency check Usability intervention Accuracy Clinical significance Statistical Motivational Conversation Timeframe Clinical tests soundness interview flow Cross-modal association Base model Overall Base model Verifiable Working Base model Alignment Goal DS agent performance DE agent Comprehensiveness sources Alliance setting HC agent c Agent Assessment goal Evaluator Scale (total annotations) Human hours 354 Plans × 2 agent conditions DS.1: statistical analysis plan 47 (708 results) DS 173 Tests (25 queries) × 2 agent conditions (346 results) 75 DS.2: code generation 2,145 MCQs × 2 agent conditions DE.1: clinical reasoning — (4,290 results) 1,511 User symptoms × 2 agent conditions DE.2: differential diagnosis (3,022 diagnoses) — DE 50 Queries × 2 agent conditions × 17 raters DE.3: medical personalization and 13 (1,700 annotations) contextualization DE.4: multimodal health data 30 Personas × 2 agent conditions × 5 raters (300 annotations) 77 interpretation and reasoning HC.1: coaching quality 31 Raters (multi-turn) × 2 agent conditions (62 annotations) 48 (evaluated by end users) HC HC.2: coaching quality 31 Interactions × 2 agent conditions x 3 raters (186 annotations) 90 (evaluated by health experts) P.1: Multi-agent system 45 Raters (multi-turn) × 3 agent conditions (135 annotations) 90 (evaluated by end users) P.2: Multi-agent system 45 Interactions × 3 agent conditions PHA (405 annotations) 140 (evaluated by health experts) 50 Personas × 3 agent conditions P.3: Interacting multi-agent system × 20 generalists + 5 experts 770 (evaluated by users and experts) (3,750 annotations) Fig. 1 | Overview of the PHA. a, The anatomy of personal health needs Gemini 2.0 family. c, Our work represents the most comprehensive evaluation with a user-centred design process. b, Design a multi-agent system, powered of a health agent thus far (more than 7,000 annotations and 1,350 h of effort by state-of-the-art LLMs, consisting of specialist DS, health DE and HC on 11 benchmarks). MCQ, multiple-choice question. agents to address diverse consumer health needs. ‘Base model’ refers to the Nature | www.nature.com | 3 Article a DS agent output The median daily step count in your DS agent data is 7,971 steps. On days when you No took more steps than this median, which I defined as your more active User query days, the average deep sleep you got Executes was about 90.0 minutes with a standard Analysis plan Code generation Do I get more deep sleep the successfully? deviation of approximately 22.4 generation and execution days that I stay more active? minutes. Yes On days when you took fewer or equal Personal health data steps than the median (less active days), the average deep sleep was (1) Identifying conceptual def analysis(summary_df: variables in the query and data pd.DataFrame, about 89.0 minutes with a standard activity_df: pd.DataFrame, deviation of approximately 19.2 (2) Formulating the correct data profile_df: pd.DataFrame, transformation steps including population_df: pd.DataFrame, minutes.The correlation between your filtering relevant subsets of ) -> Dict[str, Any]: daily step count and the minutes of data, data cleaning, joining and aggregation # First check for data sufficiency deep sleep is approximately 0.14. (3) Recognizing and addressing and filter based on timeframe missing and insufficient data past_year = datetime.now() - timedelta(days=365) (4) Applying appropriate filtered_df = statistical tests based on summary_df[summary_df.index >= user’s intent past_year] ... b Binary rubric questions of plan generation Error rate for code generation (higher is better) (lower is better) Calculation or Data availability Data logic errors check* transformation* 100% 30% 25% 80% 20% 60% 15% 40% 10% 20% 5% Statistical Data handling Output content Timeframe* tests errors* errors Overall Base Gemini Alignment* performance* General DS agent programming errors Fig. 2 | Schematic and evaluation results of the DS agent. a, The architecture to revise the code on the basis of execution errors, the original query and of the DS agent, which first generates a detailed, natural language statistical the analysis plan. b, Results of the performance evaluation of the DS agent, analysis plan from a user’s query and a data schema summary (see Supplementary demonstrating significant improvements over the base Gemini model in both Section C.2 for prompting details). Subsequently, it produces and executes analysis plan quality and code generation pass rates. Supplementary Section C.3 code to implement this plan. An iterative self-correction loop allows the agent summarizes the evaluation details and rubrics. Statistical indicator: *P < 0.05. As shown in the left panel of Fig. 2b, the DS agent significantly out- The DS agent improves analysis and coding performed the baseline overall (average score of 75.6 ± 1.4% versus The DS agent is designed to address a critical gap in existing systems: 53.7 ± 1.8%, P < 0.001, r = 0.925, Wilcoxon signed-rank test). The most the ability to interpret ambiguous, open-ended user health queries (for notable gain was on data availability, in which the near-failure baseline example, ‘am I getting more fit?’) and translate them into statistically rate of 5.3 ± 1.6% rose to 66.5 ± 3.7%, reflecting a well-documented LLM sound, executable analyses. The agent uses a two-stage architecture weakness in detecting missingness and artefacts47. Major gains were that first generates a structured, natural language analysis plan and then also observed on timeframe, data transforms and alignment, whereas translates that plan into an executable Python code with an iterative statistical tests showed no significant difference (a category in which self-correction loop (Fig. 2a). To validate this design, we independently the baseline already performed strongly). Per-dimension statistics are evaluated the performance of the agent on these two core tasks: analysis reported in Supplementary Table 5. plan generation and code generation. Code generation and execution Analysis plan generation We evaluated code generation against a benchmark of 173 unit tests We assessed the analysis plan quality by the DS agent using an created by 7 data science experts across 25 challenging query–plan expert-defined rubric covering six pre-specified dimensions of pairs (Methods; Supplementary Section C.3.2). The DS agent achieved methodological soundness (timeframe, data transforms, data suf- a 75.5 ± 3.3% first-attempt pass rate, significantly outperforming the ficiency, summary statistics, statistical tests and overall alignment; base Gemini’s 58.4 ± 3.7% (χ2 = 26.3, P < 0.001, McNemar’s test), and full definitions in the Methods, DS agent task 1). To enable scal- rose to 79.0% after up to five iterative self-correction trials. The larg- able evaluation, we developed an LLM-based autorater validated est improvement was on data-handling errors (which typically render against 354 annotations from 10 expert data scientists (intra- the entire pipeline unusable) dropping from 25.4 ± 3.3% to 11.0 ± 2.4% class correlation coefficient(3,1) = 0.838; details in Supplementary (χ2 = 12.3, P < 0.001). Per-error-category breakdowns are in Supple- Section C.3.1). mentary Table 6. We further verified that these gains generalized to 4 | Nature | www.nature.com other SOTA LLMs, indicating that the DS agent architecture is model DE agent summaries on comprehensiveness (93.3%, P < 0.001) and agnostic (see Supplementary Section C.5.3). trustworthiness (82.4%, P < 0.001; Supplementary Table 18). On overall clinical usefulness, 65% of DE agent summaries were rated ‘useful’ or ‘very useful’ versus 30% for the base model (P < 0.001; Supplementary The DE agent synthesizes multimodal insights Table 19). The expert safety review of the summaries by the DE agent The DE agent functions as a specialized cognitive layer designed to showed that they largely have no or minimal potential harm (83%), synthesize, contextualize and validate health information against and although they are non-trivial, 15% have mild harm and 2% have authoritative sources and the personal data of a user. It uses a multi-step moderate-to-severe harm (Supplementary Figs. 5 and 6). We highlight reasoning architecture that uses a suite of tools, including web search, this concern in the ‘Discussion’ section. the National Center for Biotechnology Information (NCBI) API and the DataCommons API, to iteratively investigate user queries and ground its responses in verifiable evidence (Fig. 3a). We systematically evaluated The HC agent for expert-aligned coaching the performance of the DE agent across its four foundational capabili- The HC agent is designed for robust coaching on goal setting and ties: factuality of health information, diagnostic reasoning through wellness planning, navigating multi-turn, mixed-initiative dialogue. multi-turn conversations, contextualization and personalization, and Six expert-derived design themes emerged from a formative inter- multimodal data synthesis. view study with 11 health experts: (1) goal and purpose identification, (2) active listening, (3) context clarification, (4) user empowerment, Factuality of health information (5) personalized and concrete coaching recommendations, and (6) user We benchmarked factuality on more than 2,000 expert-level multiple- feedback incorporation (Methods). The agent uses a modular archi- choice questions from board certification and coaching examinations tecture grounded in motivational interviewing (Fig. 4a), separating across endocrinology, cardiology, fitness and sleep medicine. The a core personalized coaching module from specialized conversation DE agent improves the accuracy of the base Gemini across nearly all flow modules. We evaluated the agent through a dual-perspective study domains, with overall 83.6% versus 81.8% (χ2 = 9.51, P = 0.002, McNe- with both end users and health coaching experts. mar’s test, OR = 1.67; specific results are in the Methods, DE agent task 1; Supplementary Table 11). End-user evaluation of coaching We recruited 31 participants for blinded, counterbalanced sessions Accuracy of diagnostic conversations comparing our HC agent with a baseline built on the same base Gemini We evaluated the ability of the agent to generate a differential diagnosis model (Methods, HC agent task 1). Across the chatbot usability ques- from common, self-reported symptoms. On a retrospective dataset of tionnaire, the Working Alliance inventory, the client evaluation of moti- 2,000 real-world consumer health cases, the DE agent significantly vational interviewing and SMART goal-setting instruments (Fig. 4b), outperformed a Gemini 2.0 Pro-based state-of-the-art DDx agent4 the HC agent showed significant advantages on conversation flow, (P < 0.001, r = 0.156, Wilcoxon signed-rank test), reaching a top one motivational interviewing and feedback handling (all P ≤ 0.05, GLMM accuracy of 46.1% (∆ = 4.7% over the previous SOTA), with consistent with log link function), and was preferred over the baseline on five of six gains at the top five and top ten (Supplementary Table 12 and Supple- themes (win rates of 61–71%; significant advantage on active listening, mentary Fig. 4). Similar to the DS agent, we further verified that these P = 0.025; Supplementary Table 27 perspectives and Supplementary gains generalized to other SOTA LLMs, indicating that the DE agent Table 28). The HC agent also produced more natural conversation architecture is model agnostic (see Supplementary Section D.3.3). endings (83.9% versus 74.2%). The one exception was progress meas- urement, for which the baseline scored higher. We explicitly address Personalized answers to health queries this in the integrated PHA design. We curated a set of 50 personalized health-related questions based on the query set in the previous user-centred design analysis across Expert validation of coaching diverse demographics and health conditions. In a blinded study, 17 Three health experts independently re-evaluated the end-user conver- end users compared DE agent and baseline Gemini responses on these sation transcripts using the end-user rubrics plus a specialized artificial contextualized health queries using Boolean rubrics and side-by-side intelligence coaching metrics instrument (Methods, HC agent task 2; rankings (Methods, DE agent task 3; Supplementary Table 9). On binary Supplementary Section E.2.3). The HC agent outperformed the baseline rubrics (Fleiss’ K ≥ 0.684; Fig. 3b), the most consequential gap was on on nearly every coaching dimension (∆ = 7–16%, all P < 0.001 except clar- trustworthiness (96.9 ± 0.8% versus 38.7 ± 3.3%, P < 0.001, β = 6.873, ity; see Fig. 4c, Supplementary Tables 29 and 30). The expert-perceived binomial generalized linear mixed model (GLMM)); relevance to query advantage was substantially larger than the end-user-perceived advan- and groundedness in data were comparable across systems (Supple- tage (relative ∆ = 10.2 ± 1.2% versus 3.5 ± 2.5%), and experts preferred the mentary Table 15). For side-by-side, end users preferred the DE agent HC agent across all six HC themes (win rates of 64.5–72.0%, all P < 0.05). on personalization and contextualization (71.9 ± 1.5%, P < 0.001) and As with end users, experts gave the baseline a slight edge on progress credibility of citations (76.9 ± 1.5%, P < 0.001), see Supplementary measurement, motivating the modular refinement that we apply in the Table 16 for details. integrated PHA system. A notable rater discrepancy emerged on user empowerment, for which experts preferred the HC agent (69.9% win Multimodal clinical health summaries rate), whereas end users ranked the two systems similarly (48.4%). This We assessed the ability of the DE agent to synthesize unstructured observation reinforces the value of incorporating both perspectives text, structured wearable data and PDF laboratory results from 30 in the evaluation. WEAR-ME participants into comprehensive health summaries. A panel of five expert clinicians independently evaluated summary quality in a blinded side-by-side comparison (Supplementary Table 10). On binary PHA delivers holistic health support rubrics, the DE agent significantly outperformed the base model on To address the multifaceted nature of personal health, we hypothesized clinical significance (96.4% versus 73.8%), cross-modal association that a synergistic collaboration between our specialized agents would (79.4% versus 50.0%), comprehensiveness (96.4% versus 64.2%) and deliver a superior user experience than monolithic or non-collaborative citation credibility (77.6% versus 42.7%; all P < 0.001, binomial GLMM; architectures. We therefore developed the PHA, an integrated sys- Supplementary Table 17). In direct comparisons, clinicians preferred tem guided by four core design principles: address comprehensive Nature | www.nature.com | 5 Article a DE agent output I see that you have type 1 diabetes. According to the American Diabetes Association, the target blood DE agent glucose level for adults with type 1 User query diabetes 2 hours after the start of a meal should be less than 180 mg dl–1. [Truncated What should be my target No due to space constraints]… blood glucose 2 hours In addition to medication (insulin), lifestyle after a meal? Reasoning Investigation Examination Finished? factors have a significant role in achieving postprandial glucose targets. Your Yes wearable data suggest that you currently Personal health data do not meet the general guidelines recommended by the American Heart Association [2]. [Truncated due to space Google web search constraints]… References and citations DataCommons — health focus DE agent toolbox • American Diabetes Association Health measure range https://diabetes.org/living-with-diabetes/treatment- compare care/checking-your-blood-sugar • American Heart Association Guide on NCBI API (medical research) Physical activity: https://www.heart.org/en/healthy- living/fitness/fitness-basics/aha-recs-for-physical- activity-in-adults b Binary rubric questions Side-by-side comparisons 100 Personalization and 28.1% 71.9% contextualization* 80 Accuracy (%) 60 Defining medical terms 39.6% 60.4% 40 Credibility of citations* 23.1% 76.9% 20 0 0 20 40 60 80 100 Relevance to Groundedness Trustworthiness* query in data* Wins (%) Base Gemini DE agent Base Gemini DE agent c Binary rubric questions Side-by-side comparisons Clinical consistency Comprehensiveness* 6.7% 93.3% Accuracy Trustworthiness* 17.6% 82.4% Clinical significance* 100% 0 20 40 60 80 100 80% Wins (%) 60% 40% 20% Base Gemini DE agent win Clinical soundness Usefulness before seeing a clinician Cross-modal DE agent* 20.0% 15.0% 65.0% association* Base Gemini 30.0% 40.0% 30.0% Verifiable sources* 0 20 40 60 80 100 Comprehensiveness* Wins (%) Not useful Neutral Useful or very useful Base Gemini DE agent Fig. 3 | Schematic and evaluation results of the DE agent. a, Overview of the multimodal dimensions in single-turn, contextualized question and answer workflow for the DE agent, which takes the query and personal health data of a (Q&A). c, Clinician evaluation (n = 5) of multimodal health summaries shows the user as input. The agent uses an iterative process of reasoning, investigation same conclusions. Supplementary Section D.2 summarizes the evaluation and examination, using a toolbox with access to specialized resources (see details and rubrics. The error bars indicate the standard error of the mean; Supplementary Section D.1 for prompting details). b, End-user evaluation statistical indicator: *P < 0.05. (n = 17) of the DE agent significantly outperforms a base Gemini model in user needs, adaptive support, low user burden and simplicity. The We evaluated this collaborative architecture against two strong base- framework by the PHA features a central orchestrator that manages lines: a capable single-agent system (personal health insights agent a sequential, iterative workflow involving user need understanding, (PHIA)) based on recent previous work2, and a parallel multi-agent dynamic agent task assignment, query reflection and a persistent system in which the three specialist agents respond independently. memory module to maintain conversational context (Fig. 5a). To compare the three systems, we recruited 45 end users with diverse 6 | Nature | www.nature.com a HC agent output HC agent Few shot Conclusion Recommendation User query examples module module It is completely understandable to feel that way! A lot of people are overwhelmed by social media. Open-ended No No I feel overwhelmed by social Conclude the Provide a Context Feedback media. How can I set healthier + conversational recommendation? To help you set some healthier conversation? gathering incorporation boundaries, can you tell me a bit boundaries? flow more about why you feel Yes overwhelmed? Yes Personal health data Is it the amount of time you spend on it, the content you're seeing or Structured context: user goals, constraints, something else entirely? For preferences, feedback, summary of interactions example, do you feel pressure to keep up or is it FOMO (fear of Personalized coaching module missing out)? b User experience rubric questions Side-by-side comparisons Feedback handling Goal and purpose identification (HC1) 35.5% 64.5% (CUQ)* Working Alliance Content (WAI) understanding Active listening (HC2)* 29.0% 71.0% (CUQ) 80% Context clarification on priorities and 38.7% 61.3% 64% constraints (HC3) 48% 32% User empowerment (HC4) Behavioural 16% motivation Conversation flow 51.6% 48.4% (CEMI)* (CUQ)* Personalized and concrete 38.7% 61.3% recommendations (HC5) Confidence Goals attainability User feedback incorporation (HC6) 32.3% 67.7% empowerment and alignment (CEMI) (SMART)* Progress 0 20 40 60 80 100 measurement Wins (%) (SMART)* Base Gemini win HC agent win Base Gemini HC agent c Expert evaluation rubric questions Side-by-side comparisons Conversation Personalized Goal wrap up intervention Motivational interview skill 40.9% 59.1% identification (AICM)* (HC5, AICM)* Information (HC1)* gathering (AICM)* Health recommendation quality 46.2% 53.8% Active listening Rapport (HC2)* development (AICM)* Agent credibility 33.3% 66.7% Context 90% clarification 72% Clarity (HC3)* 54% (AICM) Goal and purpose identification (HC1)* 33.3% 66.7% 36% 18% Conversation Supportiveness flow (AICM)* Active listening (HC2)* 31.2% 68.8% (CUQ)* Friendliness Context clarification on priorities Content 28.0% 72.0% (AICM)* and constraints (HC3)* understanding (CUQ)* Goal attainability User empowerment (HC4)* 30.1% 69.9% Feedback and alignment handling (SMART)* (HC6, CUQ)* Progress Personalized and concrete 35.5% 64.5% Behavioural measurement recommendations (HC5)* motivation Confidence Working (SMART)* (CEMI)* empowerment Alliance User feedback incorporation (WAI)* 29.0% 71.0% (HC4, CEMI)* (HC6)* 0 20 40 60 80 100 Wins (%) Base Gemini HC agent Base Gemini win HC agent win Fig. 4 | Schematic and evaluation results of the HC agent. a, The HC agent motivational interviewing; CUQ, chatbot usability questionnaire; SMART, architecture uses a modular process with a core personalized coaching module specific, measurable, attainable, relevant and time-bound; WAI, Working and conversation flow modules (recommendation and conclusion) to adapt the Alliance inventory. c, Coach expert evaluation (n = 3) of conversation transcripts dialogue and strategically time recommendations (Supplementary Section E.1 confirms these conclusions, revealing an even more pronounced advantage shows prompting details). b, End-user evaluation (n = 31) shows that the HC agent for the HC agent from the perspectives of experts (more details provided in significantly outperforms a base model in multi-turn coaching conversations, Supplementary Section E.2). AICM, artificial intelligence coaching metrics. leading to higher user engagement and preference. CEMI, client evaluation of Statistical indicator: *P < 0.05. Nature | www.nature.com | 7 Article a Agent collaboration Your data indicate a weak correlation between your daily steps or active zone minutes and sleep score. This suggests that simply boosting these specific metrics may not be the direct path to a significantly better sleep score. User query However, exercise and sleep are related. Main agent Research shows that consistent, How is my physical activity moderate-intensity physical activity can level related to my sleep increase total sleep time, improve sleep Orchestrator quality? Do I need to exercise (1) Infer main and supporting agent(s) Multi-agent efficiency and lessen the frequency of more? + (2) Determine collaborative workflow collaboration Self-reflection waking up during the night, leading to a (3) Rephrase of queries for each agent better overall quality of rest. Personal health data Supporting agent(s) So, although more exercise might not be the only answer, it is still a good idea to be active. To get a better picture, could you Memory update tell me a bit about your current constraints of the exercise? For example, what kind of daily routines do you follow, how much Step 1: user need Step 2: agent Step 3: query Step 4: memory time you can allocate for exercise? understanding orchestration reflection update b End-user evaluation of conversations Preference ranking Natural conversation Personalized (HC)* Multi-agent PHA 22.2% 31.1% 46.7% coaching (HC)* Domain knowledge Multi-agent (parallel) 33.3% 37.8% 28.9% integration (DE) Single-agent PHIA 44.4% 31.1% 24.4% User 100% empowerment 80% (HC)* 60% 0 20 40 60 80 100 40% Wins (%) 20% Data science Ranked 3 Ranked 2 Ranked 1 analysis (DS)* Health data End-user response quality ranking interpretation (DS + DE) Overall quality Multi-agent PHA 26.7% 28.9% 44.4% (DS + DE + HC)* Personalized Multi-agent (Parallel) 26.7% 42.2% 31.1% suggestions Grounded suggestions (DS + HC) (HC+DE) Single-agent PHIA 46.7% 28.9% 24.4% 0 20 40 60 80 100 Single agent Multi-agent Multi-agent Wins (%) (PHIA) (parallel) (PHA) Ranked 3 Ranked 2 Ranked 1 c Expert evaluation of conversations Response quality ranking Appropriate User data health interpretation information Accurate health Multi-agent PHA* 17.8% 15.6% 66.7% (DS + DE)* (DE)* information (DE)* Engagement Multi-agent (Parallel) Appropriate user 25.2% 54.1% 20.7% (HC)* data (DS)* User empowerment Single-agent PHIA 57.0% 30.4% 12.6% (HC)* Overall quality (DS + DE + HC)* 0 20 40 60 80 100 Conciseness and Wins (%) clarity (HC)* Ranked 3 Ranked 2 Ranked 1 Ask clarifying questions (HC)* Appropriate timing (HC)* Red flag Ranking of effectiveness and usefulness identification (DS + DE + HC)* Feedback incorporation (HC)* Multi-agent PHA* 16.3% 21.5% 62.2% Effective synthesis Guided (DS + DE + HC)* Multi-agent (Parallel) 28.9% 45.9% 25.2% goal setting (HC)* Coherent and consistent Informed Knowledge (DS + DE + HC)* Single-agent PHIA 54.8% 32.6% 12.6% suggestions integration (DS + HC)* (HC + DE)* 0 20 40 60 80 100 Single agent Multi-agent Multi-agent Wins (%) (PHIA) (parallel) (PHA) Ranked 3 Ranked 2 Ranked 1 Fig. 5 | Schematic and evaluation results of the PHA. a, Overview of the conversations shows the PHA as the modal top choice over both single-agent collaborative workflow for the PHA. An orchestrator dynamically assigns tasks and parallel multi-agent baselines for overall quality and helpfulness. c, Health to the DS, DE and HC agents, managing a multi-step process of collaboration, expert evaluation (n = 3) of these live conversation transcripts enhances these reflection and memory update to synthesize a cohesive response (see conclusions, revealing a substantially more pronounced preference for the Supplementary Section F.1 for prompting details). b, End-user evaluation PHA system over the baselines. Statistical indicator: *P < 0.05. Supplementary (n = 45, matched with WEAR-ME-derived personas) of live multi-turn Section F.2 summarizes more evaluation details. 8 | Nature | www.nature.com health conditions, covering all WEAR-ME persona profiles (see Sup- plementary Table 35), as well as 3 health experts (Methods, PHA tasks). Relative computational cost advantage We also compared the number of LLM calls and response time across End users prefer collaborative PHA agents. Despite its iterative orchestration, the PHA requires fewer Figure 5b summarizes the results from our end-user evaluation. The LLM calls and responds faster than the parallel multi-agent baseline findings indicate that although all three systems were perceived as (see Methods, ‘Computational cost’ section), demonstrating that the generally competent, our proposed PHA architecture was consistently dynamic agent assignment by the PHA delivers superior user experience favoured for providing a better overall experience. End users ranked with greater computational efficiency than a static parallel approach. PHA as their top choice nearly half the time (46.7% on overall prefer- ence and 44.4% on response quality), more frequently than the parallel Overarching summary multi-agent (28.9% and 31.1%, respectively) and single-agent PHIA (24.4% Across the specialist and integrated evaluations, the modular sub- and 24.4%, respectively) baselines. The detailed rubric breakdown shows agents improved their targeted capabilities, and the orchestrated PHA that PHA numerically outperforms both baselines on every rubric, with outperformed the single-agent and parallel multi-agent baselines. the advantage reaching post-hoc statistical significance on personal- Together, these findings support a modular, collaborative architecture ized coaching (PPHA–PHIA < 0.001), natural conversation (P = 0.002), user that emulates the collaborative and specialized nature of human expert empowerment (P = 0.032) and overall quality (P = 0.044, all GLMM with teams to support personal health and well-being. log link function and post-hoc Tukey’s honest significance test), as well as a PHA-versus-parallel advantage on data science analysis (P = 0.016). Discussion Per-rubric statistics are reported in Supplementary Tables 36 and 37. PHA advantages and user journeys Beyond the ranking and scores, end users qualitatively praised the coher- The PHA assigns data analysis, health-domain reasoning and coaching ent and personalized conversation flow of the PHA. For instance, one to specialist sub-agents coordinated by an orchestrator. This modu- participant stated that the PHA “did a really stellar job giving me person- larity allows each specialist capability to be developed and evaluated alized advice based off of what I was asking and off of my health scores independently and enables agents or base models to be replaced and info, and it really honed in on a routine change that was unique without redesigning the overall workflow. Personal health questions to me and simple to fit into my current lifestyle and evening routine”. often combine quantitative interpretation, biomedical context and Another participant noted the PHA as “personable and knowledgeable behavioural change guidance; the orchestrator can recruit the relevant and knew your history and dove right into it”. Although the baseline specialists and synthesize their outputs into one response. In the inte- agents may possess the skills on domain knowledge and coaching, the grated evaluation, the PHA received higher ratings than the baselines on iterative framework of the PHA allows the HC and DE agents to reflect complex queries from both end users and experts: differences relative on and contextualize the specific findings from the DS agent, providing to the baselines ranged from 1.6 to 15.4% (average of 4.6%) for end users a more deeply synthesized and insightful response. and 5.7 to 39.0% (average of 17.1%) for experts. Our contribution is an architecture grounded in empirically derived user needs. We do not Experts favour the PHA framework aim to claim that our configuration is universally optimal. By releasing The evaluation from health experts, summarized in Fig. 5c, reveals the PHA implementation and evaluation benchmark, we intend this a more pronounced advantage for the PHA system than the results work as a reusable foundation for the community to critique, swap in derived from end users. Coach experts identified major quality dif- different agents or base models, and build stronger PHAs. ferences that, although not immediately apparent to end users, are Our analysis shows that the PHA demonstrates strong competency fundamental to providing safe and robust guidance and critical for the across all four CUJs, with particular strength over existing systems in long-term effectiveness and user success of the system. Experts gave personal health data interpretation (CUJ2) and generating wellness PHA the highest overall quality rating (72.8 ± 1.4% versus 64.3 ± 1.3% advice (CUJ3). The framework excels at identifying patterns in wear- for parallel multi-agent and 54.4 ± 1.4% for PHIA, all pairwise P < 0.001, able data, contextualizing these findings against established health GLMM with log link function), with this advantage extending across guidelines, and producing actionable, personalized recommenda- nearly all 17 individual rubrics (Supplementary Table 38). Experts also tions. However, addressing symptom assessment (CUJ4) presents ranked PHA as the best system in 66.7% of conversations on overall a more complex and critical challenge. Although PHA outperforms quality and 62.2% on effectiveness, more than twice the preference for general-purpose models in providing basic symptom information, the parallel multi-agent (20.7% and 25.2%, respectively) and roughly its current architecture is not designed to offer definitive medical five times that of the PHIA (both 12.6%, all pairwise P < 0.001, Fried- diagnoses or replace the judgement of a qualified health-care pro- man test followed by post-hoc Wilcoxon signed-rank test with Holm– fessional. Yet simply deferring such questions to clinicians may not Bonferroni; Supplementary Table 39). The qualitative comments by address the delays that often motivate people to seek artificial intel- experts echoed these results across coaching and data science dimen- ligence support48. A future system could instead help users to record sions. One coaching expert observed the PHA “consistently excelling at symptoms, identify patterns, prepare questions and navigate care the behavioural aspects of motivational interviewing and… SMART goal while avoiding diagnostic claims1. Such a preparatory role requires creation”, whereas another highlighted the data-science foundation prospective validation and safeguards against premature or inaccurate of the PHA, “synthesizing Fitbit metrics and lab results into a baseline suggestions. Supplementary Discussion G provides additional inter- narrative… [to] provide a necessary mirror for the user”. pretation and future directions for each single-agent and multi-agent collaboration. Scaling with synthesized conversations To complement the live evaluations that are ecologically valid but Ethical risks and future safeguards bounded by participant time, we further leveraged the 50 WEAR-ME The PHA is a research prototype for consumer wellness, not a diagnos- personas and synthesized 150 multi-turn conversations (50 × 3) tic or clinical triage system, and should not replace professional care. between persona-instructed user LLM agents and the 3 systems, Health-facing agents nevertheless risk reinforcing disparities when their and recruited a separate cohort of 20 end-user generalist evaluators models or evaluation data underrepresent populations. Future work and 5 health experts to rate them using the same rubrics. We observed should incorporate fairness metrics and evaluations across devices, consistent patterns with the live findings in tasks 1 and 2 (see details in languages, literacy levels and demographic strata. The prototype uses Supplementary Section F.3.3). in-prompt disclaimers and conservative symptom-response policies Nature | www.nature.com | 9 Article as initial safeguards. Future versions should add persistent cross-turn 6. Vert, J.-P. How will generative AI disrupt data science in drug discovery? Nat. Biotechnol. 41, 750–751 (2023). notices, detect high-risk queries, and either refuse them or respond with 7. Gottweis, J. et al. Accelerating scientific discovery with Co-Scientist. Nature 655, 487–496 an explicit handoff to a clinician so that the system does not delay care. https://doi.org/10.1038/s41586-026-10644-y (2026). These mechanisms require testing for false reassurance and unneces- 8. Khasentino, J. et al. A personal health large language model for sleep and fitness coaching. Nat. Med. https://doi.org/10.1038/s41591-025-03888-0 (2025). sary escalation, with clinician review of high-risk failures. Moreover, 9. Breda, J. et al. SymptomAI: towards a conversational AI agent for everyday symptom open-web retrieval creates a related risk of conflicting or low-quality assessment. Preprint at https://doi.org/10.48550/arxiv.2605.04012 (2026). evidence, and future deployment should restrict health retrieval to 10. WHOOP. Introducing whoop coach, powered by OpenAI. WHOOP https://www.whoop. com/us/en/thelocker/introducing-whoop-coach-powered-by-openai/ (2023). curated sources such as WHO, PubMed, NICE and CDC, and display 11. Oura Team. Introducing Oura advisor: your AI-powered personal health companion. Oura source provenance beside each recommendation. Future systems https://ouraring.com/blog/oura-advisor/ (2025). could provide layered explanations showing which data and evidence 12. Tu, T. et al. Towards conversational diagnostic AI. Nature 642, 442–450 https://doi.org/ 10.1038/s41586-025-08866-7 (2025). informed a response. Finally, the research harnesses sandbox generated 13. Zheng, N. S. et al. Sleep patterns and risk of chronic disease as measured by long-term code, but real-world deployment requires systematic code validation monitoring with commercial wearable devices in the all of us research program. Nat. Med. and standardized privacy and security controls (for example, Health 30, 2648–2656 (2024). 14. Ringeval, M., Wagner, G., Denford, J., Paré, G. & Kitsiou, S. Fitbit-based interventions for Insurance Portability and Accountability Act and General Data Protec- healthy lifestyle outcomes: systematic review and meta-analysis. J. Med. Internet Res. 22, tion Regulation). e23954 (2020). 15. Kim, J.-W. et al. Impact of personal health records and wearables on health outcomes and patient response: three-arm randomized controlled trial. JMIR mHealth uHealth 7, e12070 Limitations and conclusion (2019). Our evaluation provides evidence of conversational quality and short- 16. Jörke, M. et al. GPTCoach: towards LLM-based physical activity coaching. In Proc. 2025 term task efficacy, not sustained behavioural change or improvement CHI Conference on Human Factors in Computing Systems (eds Yamashita, N. et al) https:// doi.org/10.1145/3706598.3713819 (Association for Computing Machinery, 2025). in longitudinal health outcomes. Those outcomes require multi-session 17. Jörke, M. et al. Bloom: designing for LLM-augmented behavior change interactions. In deployments that retain previous conversation context, test behav- Proc. 2026 CHI Conference on Human Factors in Computing Systems (eds Oliver, N. et al.) ioural interventions, and measure engagement and adherence over 1–27 (Association for Computing Machinery, 2026). 18. Merrill, M. A., Tan, M., Gupta, V., Hartvigsen, T. & Althoff, T. Language models still struggle time. Longitudinal studies should recruit people responding as them- to zero-shot reason about time series. In Findings of the Association for Computational selves and test whether benefits persist with long-term use. Our current Linguistics: EMNLP 2024 (eds Al-Onaizan, Y. et al.) 3512–3533 (Association for Computational Linguistics, 2024). evaluation also relies partly on LLM-based autoraters, which can be 19. Topol, E. Deep Medicine: How Artificial Intelligence Can Make Healthcare Human Again sensitive to ordering, formatting, length and source. Autorater conclu- (Hachette, 2019). sions should remain cross-checked against blinded human judgements 20. Srinivas, V. et al. Substance over style: evaluating proactive conversational coaching agents. Preprint at https://doi.org/10.48550/arxiv.2503.19328 (2025). as models and prompts change. Moreover, evaluation on WEAR-ME 21. Graham, G., Goren, N., Sounderajah, V. & DeSalvo, K. Information is a determinant of is Fitbit centric, limiting device generalizability. In our live studies, health. Nat. Med. 30, 927–928 (2024). condition-stratified participant matching on WEAR-ME was intended 22. Basar, E., Hendrickx, I., Krahmer, E., Bruijn, G.-J. & Bosse, T. To what extent are large language models capable of generating substantial reflections for motivational to increase scenario relevance but cannot eliminate perspective-taking interviewing counseling chatbots? A human evaluation. In Proc. 1st Human-Centered bias. Future evaluation across devices, languages, literacy levels and Large Language Modeling Workshop (eds Soni, N. et al.) 41–52 (Association for demographic groups, including older adults and people with disabili- Computational Linguistics, 2024). 23. Wang, X. et al. Self-consistency improves chain of thought reasoning in language models. ties, is therefore needed, and our open benchmark provides a basis for Preprint at https://doi.org/10.48550/arxiv.2203.11171 (2022). these extensions. Finally, computational burden is a constraint. PHA 24. Chen, J., Saha, S. & Bansal, M. ReConcile: round-table conference improves reasoning averaged 6.5 LLM calls and 205.3 s per response round, compared with via consensus among diverse LLMs. In Proc. 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (eds Ku, L.-W. et al.) 7066–7085 3.0 calls and 36.4 s for single-agent PHIA (Methods), which may limit (Association for Computational Linguistics, 2024). scalability in resource-constrained settings. Optimized orchestration, 25. Tang, X. et al. MedAgents: large language models as collaborators for zero-shot medical parallel execution, caching and lighter routing models may reduce reasoning. In Findings of the Association for Computational Linguistics: ACL 2024 (eds Ku, L.-W. et al.) 599–621 (Association for Computational Linguistics, 2024). this cost. 26. Du, Y., Li, S., Torralba, A., Tenenbaum, J. B. & Mordatch, I. Improving factuality and Overall, this study evaluated DS, DE and HC agents together with reasoning in language models through multiagent debate. Preprint at https://doi. their orchestration across 11 benchmark tasks and more than 1,350 h of org/10.48550/arxiv.2305.14325 (2023). 27. Kim, Y. et al. MDAgents: an adaptive collaboration of LLMs for medical decision-making. end-user and expert assessment. The openly released implementation Neural Inf. Process. Syst. 37, 79410–79452 (2024). and benchmark allow other researchers and engineers to test different 28. Zhang, Y. et al. Benchmarking data science agents. In Proc. 62nd Annual Meeting of the agents, base models, collaboration patterns and safeguards. These Association for Computational Linguistics (Volume 1: Long Papers) (eds Ku, L.-W. et al.) 5677–5700 (Association for Computational Linguistics, 2024). results provide a basis for further research of PHAs for daily health 29. Hong, S. et al. Data interpreter: an LLM agent for data science. Preprint at https://doi. and well-being. org/10.48550/arxiv.2402.18679 (2024). 30. Gu, K. et al. BLADE: benchmarking language model agents for data-driven science. In Findings of the Association for Computational Linguistics: EMNLP 2024 (eds Al-Onaizan, Y. et al.) 13936–13971 (Association for Computational Linguistics, 2024). Online content 31. Zhang, D. et al. DataSciBench: an LLM agent benchmark for data science. Preprint at Any methods, additional references, Nature Portfolio reporting summa- https://doi.org/10.48550/arxiv.2502.13897 (2025). 32. Cheng, L., Li, X. & Bing, L. Is GPT-4 a good data analyst? The 2023 Conference on ries, source data, extended data, supplementary information, acknowl- Empirical Methods in Natural Language Processing (eds Bouamor, H. et al.) https://doi. edgements, peer review information; details of author contributions org/10.18653/v1/2023.findings-emnlp.637 (Association for Computational Linguistics, and competing interests; and statements of data and code availability 2023). 33. Yin, P. et al. Natural language to code generation in interactive data science notebooks. are available at https://doi.org/10.1038/s41586-026-11161-8. In Proc. 61st Annual Meeting of the Association for Computational Linguistics (eds Rogers, A. et al.) 126–173 (Association for Computational Linguistics, 2023). 1. Cao, Y. et al. More than decision support: exploring patients’ longitudinal usage of 34. Wu, X. et al. DACO: towards application-driven and comprehensive data analysis via code large language models in real-world healthcare-seeking journeys. In Proc. 2026 CHI generation. In The Thirty-eight Conference on Neural Information Processing Systems Conference on Human Factors in Computing Systems (eds Oliver, N. et al.) https://doi. Datasets and Benchmarks Track (eds Globerson, A. et al.) https://doi.org/10.52202/ org/10.1145/3772318.3791946 (Association for Computing Machinery, 2026). 079017-2878 (2024). 2. Merrill, M. A. et al. Transforming wearable data into personal health insights using large 35. Hu, X. et al. InfiAgent-DABench: evaluating agents on data analysis tasks. In International language model agents. Nat. Commun. 17, 1143 (2026). Conference on Machine Learning (eds Salakhutdinov, R. et al.) 19544–19572 (PMLR, 3. Lee, S. H. Natural language generation for electronic health records. npj Digit. Med. 1, 63 2024). (2018). 36. Singhal, K. et al. Large language models encode clinical knowledge. Nature 620, 172–180 4. McDuff, D. et al. Towards accurate differential diagnosis with large language models. (2023). Nature 642, 451–457 https://doi.org/10.1038/s41586-025-08869-4 (2025). 37. Nori, H. et al. Can generalist foundation models outcompete special-purpose 5. Benary, M. et al. Leveraging large language models for decision support in personalized tuning? Case study in medicine. Preprint at https://doi.org/10.48550/arxiv.2311.16452 oncology. JAMA Netw. Open 6, e2343689 (2023). (2023). 10 | Nature | www.nature.com 38. Nori, H. et al. From medprompt to o1: exploration of run-time strategies for medical 47. Gu, K. et al. RADAR: benchmarking language models on imperfect tabular data. Preprint challenge problems and beyond. Preprint at https://doi.org/10.48550/arxiv.2411.03590 at https://doi.org/10.48550/arxiv.2506.08249 (2025). (2024). 48. Kohane, I. S. Compared with what? Measuring AI against the health care we have. N. Engl. 39. Mercado, J., Espinosa-Curiel, I. E. & Martı́nez-Miranda, J. Embodied conversational agents J. Med. 391, 1564–1566 (2024). providing motivational interviewing to improve health-related behaviors: scoping review. J. Med. Internet Res. 25, e52097 (2023). Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in 40. Jörke, M. et al. Supporting physical activity behavior change with LLM-based published maps and institutional affiliations. conversational agents. Preprint at https://doi.org/10.48550/arxiv.2405.06061 (2024). 41. Mantena, S. et al. Fine-tuning large language models in behavioral psychology for scalable physical activity coaching. npj Cardiovasc. Health 2, 48 https://doi.org/10.1038/ Open Access This article is licensed under a Creative Commons Attribution s44325-025-00083-5 (2025). 4.0 International License, which permits use, sharing, adaptation, distribution 42. Tran, K.-T. et al. Multi-agent collaboration mechanisms: a survey of LLMs. Preprint at and reproduction in any medium or format, as long as you give appropriate https://doi.org/10.48550/arxiv.2501.06322 (2025). credit to the original author(s) and the source, provide a link to the Creative Commons licence, 43. Metwally, A. A. et al. Insulin resistance prediction from wearables and routine blood and indicate if changes were made. The images or other third party material in this article are biomarkers. Nature 652, 451–461 (2026). included in the article’s Creative Commons licence, unless indicated otherwise in a credit line 44. The Gemini Team. Gemini: a family of highly capable multimodal models. Preprint at to the material. If material is not included in the article’s Creative Commons licence and your https://doi.org/10.48550/arxiv.2312.11805 (2023). intended use is not permitted by statutory regulation or exceeds the permitted use, you will 45. Saab, K. et al. Capabilities of Gemini models in medicine. Preprint at https://doi.org/ need to obtain permission directly from the copyright holder. To view a copy of this licence, 10.48550/arxiv.2404.18416 (2024). visit http://creativecommons.org/licenses/by/4.0/. 46. Yang, L. et al. Advancing multimodal medical capabilities of Gemini. Preprint at https:// doi.org/10.48550/arxiv.2405.03162 (2024). © The Author(s) 2026 Nature | www.nature.com | 11 Article Methods Consumer health queries. We compiled and analysed consumer health queries from Google Search, Gemini and the Fitbit community Real-world wearable and health data forum, from January 2024 to February 2025. In addition, in October To ground our agent evaluation with real-world validity, we evaluated 2024 through February 2025, we launched Insights Explorer through the applicability and robustness of the framework against the WEAR-ME Fitbit Labs49. Participation was entirely voluntary and required users study, a large-scale (n = 1,165) effort to investigate metabolic health43. to explicitly opt-in and provide in-app consent for their data to be used The overall study design is outlined in Extended Data Fig. 9a. As part for research and development. As part of this consented experience, of WEAR-ME, consented participants were asked to link their Fitbit participants were able to ask health queries and receive responses account to Google Health Studies, authorizing the collection of their from a prototype generative artificial intelligence (AI) conversational wearable data for the duration of the study and for up to 3 months agent. We composed a set of 1,370 de-identified user queries that rep- before they joined. Once enrolled, participants were asked to (1) com- resent a wide range of questions regarding daily health and wellness plete questionnaires covering demographics, health history and per- from real-world users. Supplementary Table 1 lists some examples of sonal health insights, (2) schedule a blood draw with a quest patient these queries. service centre, and (3) wear their Fitbit or Pixel Watch throughout the day and night (for at least 3 out of every 4 days). In total, WEAR-ME Fitbit Labs survey data. To enrich the query data, we also collected contains multimodal data from each participant on their question- survey data from participants (n = 555) in the Insights Explorer Fitbit naire answers, Fitbit records and blood test results. More details of Labs to ask about their personal health goals and sentiments when the dataset are described in Supplementary Section B.2. The WEAR-ME interacting with the agent. Specifically, the survey includes questions study was conducted in full compliance with privacy regulations, with about user needs (for example, ‘what are you hoping to achieve by the study approved by the Advarra Institutional Review Board (IRB; using [the prototype]?’, ‘which of the following aspects of health and #Pro 00074093). All participants provided informed e-consent for wellness, if any, did you explore in [the prototype]?’), user experience their data to be used in research and publications. and satisfaction (for example, ‘indicate any aspects of [the prototype] Throughout our evaluation, we leveraged the WEAR-ME dataset for that have/have not met your expectations’), engagement and retention various evaluation tasks (see tasks DS.1, DE.3, DE.4, HC.1, HC.2, P.1, P.2 (for example, ‘what was the main functionality or benefits that you and P.3 in Fig. 1c). As introduced later in this section, we contextualized continued using [the prototype] for?’) and other user characteristics our end-user and expert evaluations on a subset cohort of WEAR-ME (for example, previous experience with wearable sensing, AI and con- for real-world validity. Working with clinicians, we identified the ten versational agents). Supplementary Table 1 lists the details of these most common health profiles among the participants and then ran- laboratory surveys. domly selected five individuals representing each of those conditions (see the specific ten profiles and persona examples in Supplementary Expert workshop. To complement users’ perspective, we further con- Section B.2). These 50 user personas were used to prepare evaluation ducted a 1-h critical user journey workshop with experts across user samples for our agents’ multiple evaluation tasks (for example, task 3 experience, product, research and engineering (n = 14) to categorize and task 4 for the DE agent, and task 1, task 2 and task 3 for the PHA). the data collected. The workshop procedure focused on summarizing The construction of the evaluation cohort is depicted in Extended the user journey goals and the specific associated tasks. After collecting Data Fig. 9b. multiple experts’ perspectives, we adopted a standard thematic analy- sis process50. Two researchers independently coded the same subset Choice of base language model of these queries and all survey data to jointly develop a codebook of We used the Gemini 2 family (Flash/Pro) of models44 for our experi- hierarchical user journeys (categories and sub-categories). The same ments. Gemini has demonstrated strong performance on medical researchers then categorized the rest of the user queries into these tasks8,45,46. To demonstrate that our method is model agnostic, we categories and expanded the codebook as needed. further evaluated our agent architecture on other LLM families across Our analysis of user needs identified four CUJs, spanning data inter- OpenAI’s GPT and Anthropic’s Claude (see result tables in Supplemen- pretation, knowledge seeking, wellness advice and symptom assess- tary Data). In this work, our focus was not to train a new LLM but rather ment (see results). Satisfying these diverse requirements demands to design and construct the agentic system around a given base LLM. more than a single monolithic system and calls for a set of specialized, By selecting Gemini as the base model, we could ensure that (1) there complementary capabilities. was no training data contamination, and (2) no health data from real Our agent development approach is therefore grounded in both the participants would be logged by the model and used for training future cognitive theory of modular cognition51, which posits that the mind uses models. Although existing contamination was not a main concern, as a specialized subsystems for different tasks, and the empirical success majority of our analyses involve novel datasets that are unlikely to be of human health teams, in which experts with complementary skills part of any LLM training dataset, choosing Gemini allowed us to verify collaborate. Inspired by this, we deconstructed the problem space and that any testing of the models did not lead to data entering a future propose a multi-agent personal health assistant that brings together training set. We note that our design of sub-agents and the agent teams three core roles, each embodied by a specialized sub-agent: data analy- are agnostic to the base LLM. We expect that our proposed solutions sis, health domain expertise and behavioural coaching, working in will be generalizable to other LLMs. concert to emulate a professional health support team. User-centred design of the PHA DS agent development We adopted a user-centred approach to identify and categorize the This agent mainly focuses on addressing user queries about personal needs of end users for a PHA. Our process integrated insights from data. Existing research has explored the ability of LLMs to answer data three layers of complementary data collection: real-world user queries, science questions2,28–35, but their capacity to interpret open-ended, targeted survey responses and expert synthesis. Together, these layers personalized data inquiries and perform statistically sound data analy- form a coherent pipeline: we began by capturing broad, naturalistic ses remains unclear, and there is a lack of open benchmarks assessing user intent through consumer health queries; we then deepened this their reliability in this area. We developed the first personal open-ended understanding through structured survey data that elicited the goals data science agent capable of analysing both personal and population- and satisfaction of users. Finally, we conducted expert workshops to level time-series wearable data to provide numerical health insights. distill and organize these findings into actionable requirements. Our agent enables users to ask diverse, open-ended and personalized questions about their health data and provide meaningful, data-driven The first stage translates an open-ended natural language query responses, filling a critical gap in personalized data analysis. We also into a precise, structured analysis plan. To ground its reasoning, the open-sourced an evaluation benchmark for these capabilities. Example planner is provided with a rich contextual summary of the available conversations between a user and the DS agent are shown in Supple- data, including table schemas, column descriptions with data types mentary Fig. 1. and semantic tags, and summary statistics56. The output is a structured We focused on a set of capabilities for the DS agent that are useful, natural language paragraph that explicitly outlines the conceptual yet tractable to evaluate2,30,34. These capabilities are informed by the variables, required data transformations (for example, filters, joins and types of user interactions and data that are expected in the context of aggregations), and the chosen statistical model, directly instantiating the CUJs (presented in the results). Broadly, we define the capability the components listed above. of the DS agent to address queries that involve analysis of numerical The second stage receives this structured plan and is tasked with data (for example, wearable streams and medical records), with the generating executable code. This focus on a well-defined specification goal as obtaining statistical outcomes that align with the query intent allows the LLM to leverage its strong code generation capabilities55. The and context surrounding those statistics, such as timeframe, type of generated code is then executed in a sandboxed environment. We used statistics, assumptions, among others. Specifically, the capabilities of an iterative refinement loop: if execution fails due to syntactic errors the agent are as follows. or runtime exceptions, the agent re-invokes the stage with the original query, plan and the error message as feedback. This self-correction Decompose an open-ended query into a structured analysis plan. cycle, which has been shown to improve robustness57, continues until For broad or ambiguous queries often posed by end users without bug-free execution is achieved and a numerical result is produced. data science expertise, the DS agent should translate them into appro- Figure 2a illustrates the architecture enabling these functions in the priately specific, unambiguous, tractable queries and analysis plans DS agent. The details of agent prompt and architecture are included in by: Supplementary Section C.2. (1) Identifying conceptual variables52, which recognizes the concepts in the question that can and should be represented in the data (for DE agent development example, representing fitness as average daily steps and resting This agent aims to answer any queries that are related to medical heart rate for a query such as ‘have I gotten more fit over the past knowledge or daily symptoms. Although previous work has evaluated few months?’). the medical knowledge of LLMs36–38, many essential skills of an ideal (2) Formulating the correct and exact data transformation steps to domain health expert are underexplored, such as interpreting data operationalize the variables across multiple tables of user data and within contexts, for example, population and environmental factors, population-level data tables. This involves the following optional and performing summarization and reasoning across data modalities steps: from wearables and medical records. We built a DE agent equipped with (a) Filtering relevant subsets of data (for example, filter for activities a suite of tools to address these challenges. The agent integrates the in the past 3 months) wearable data of users, medical records and contextual data to generate (b) Cleaning messy data (for example, handling missing or mis- accurate, relevant responses to complex health inquiries. It advances recorded wearable data) the state-of-the-art LLMs by providing nuanced multi-modal interpre- (c) Joining multiple tables (for example, joining the activities data tations that consider the broader context of personal health. Example table with the daily summary data table on the same day) conversations with the DE agent are shown in Supplementary Fig. 2. (d) Aggregating data (for example, getting average steps and resting We designed the DE agent to function as a specialized cognitive layer heart rate grouped by month) for health applications. Its purpose is not merely to retrieve informa- (3) Recognizing, highlighting or addressing missing or insufficient tion, but to synthesize, contextualize and validate it against authori- data to prevent unreliable conclusions. tative sources and patient-specific data. By acting as a focused and (4) Applying appropriate statistical tests and calculations on the basis reliable source of medical expertise, the DE agent aims to minimize of the intent of the user. misinformation risk and translate complex data into clear, actionable insights, serving as a critical foundation for our multi-agent design. Note that given the open-ended nature of user queries, multiple (These capabilities were investigated in a controlled, academic context; interpretations and answers may be valid. Accordingly, in one single they do not indicate real-world products that require an evaluation of interaction, our DS agent is designed to provide at least one appropriate complex regulatory landscapes.) response, rather than an exhaustive enumeration of all possibilities. To fulfil this role, the DE agent needs to move beyond the simple Users can have follow-up conversations to explore other options with query–response behaviour of standard LLMs. It should be equipped the agent. In the case in which the data in user queries are not available, with a set of distinct yet interconnected skills that mirror the reasoning the agent might check the availability of the data with users and request of a human clinician. This involves not only accessing factual knowledge data from users, if needed. but also applying it within the context of the specific health status and history of an individual. To formalize this multifaceted expertise, we Compute accurate numerical results through code generation, defined and evaluated the core competencies of the agent across four execution and feedback. On the basis of a precisely specified analysis foundational capabilities: plan (that is, one without any ambiguity for alternative methods and (1) Accuracy of health information. The DE agent should provide acc- models53), the agent should be able to generate, execute and debug urate and reliable medical knowledge spanning diverse personal code that follows the exact specifications of the analysis plan, such that health and clinical domains, ensuring that information is consistent it can execute bug-free code and compute accurate results. with expert-level understanding. On the basis of the delineated capabilities, the architecture of the (2) Diagnostic reasoning acumen. The agent can perform differential DS agent is designed as a modular, two-stage pipeline: (1) analysis plan diagnostic reasoning by generating and ranking plausible diagnoses generation and (2) code generation with iterative execution54. This on the basis of the symptoms and relevant medical history of the decomposition is critical for mitigating the risk of factual inaccuracies user conveyed through natural conversations. and logical errors common in end-to-end generation for complex data (3) Contextualization and personalization efficacy. The agent should tasks55. It separates the semantic reasoning of ‘what to do’ from the contextualize and personalize health information and advice, tai- syntactic task of ‘how to do it’. loring outputs to the specific demographic profile, pre-existing Article conditions and health goals of a user to ensure relevance, clarity developed theme codebooks and iterated until convergence. and safety. This qualitative data provided a foundational understanding of (4) Multimodal data synthesis and reasoning. The DE agent should nuanced coaching expertise, leading to the following core HC agent synthesize and reason over heterogeneous data sources, generat- capabilities. ing coherent and informative summaries from complex real-world For goal and purpose identification, the HC agent needs to under- inputs that integrate wearable data (for example, simple statistics stand the goal and the motivation behind the goal of the user, maintain or complex outcomes computed by the DS agent) with medical a goal-oriented conversation, and bring the user back to the main goal records (for example, laboratory results and medical history). when they are deviating off the topic. For active listening, the agent should provide occasional summary The DE agent uses a multi-step planning, reasoning and acting archi- and paraphrasing of the state of conversation to circle back to users tecture58, as shown in Fig. 3a. The agent receives a user query, wearable and ensure correct understanding and goal alignment. data and medical records as input. The agent processes the query and For context clarification, our HC agent should collect and understand chooses actions from a set of tools that include querying: Web Search the constraints, preferences and previously attempted solutions of (https://www.google.com), the NCBI (https://www.ncbi.nlm.nih.gov) users. application programming interface (API) and the DataCommons API For user empowerment, the HC agent needs to empower the user by (https://datacommons.org). A Python Sandbox is also available for building trust, guiding them to navigate their own paths when explor- executing APIs. ing potential solutions, and providing supportive