Wie latente und prompting-bedingte Verzerrungen in KI-generierten historischen Narrativen Meinungen beeinflussen

PNAS Nexus, 2026, 5, pgag022 https://doi.org/10.1093/pnasnexus/pgag022 Advance access publication 3 March 2026 Brief Report How latent and prompting biases in AI-generated historical narratives influence opinions Downloaded from academic.oup.com/pnasnexus/article/5/3/pgag022/8503065 by PricewaterhouseCoopers AG user on 24 October 2026 a b,c, b d Matthew Shu , Daniel Karell *, Keitaro Okura and Thomas R. Davidson a Department of Statistics and Data Science, Yale University, New Haven, CT 06511, USA b Department of Sociology, Yale University, New Haven, CT 06511, USA c Institution for Social and Policy Studies, Yale University, New Haven, CT 06511, USA d Department of Sociology, Rutgers University, New Brunswick, NJ 08901, USA *To whom correspondence should be addressed: Email: daniel.karell@yale.edu Edited By David Rand Abstract Large language models (LLMs) can be used to persuade people on a range of issues, particularly through user-driven strategies such as personalizing messages and dialogues intended to change minds. However, their capacity to influence opinions through subtle, latent ideological framing remains relatively understudied. We investigate whether AI-generated historical summaries affect social and political opinions through a preregistered experiment (N = 1,912). Participants read Wikipedia or GPT-4o summaries of two historical events, with AI summaries maintaining factual accuracy while exhibiting different types of framing biases. Default AI summaries led to more liberal opinions compared with Wikipedia, demonstrating the persuasive capability of LLM’s latent biases. Summaries purposefully induced with a liberal framing also led to more liberal opinions, regardless of readers’ ideologies. Summaries constructed with a conservative framing produced conservative shifts primarily among conservative readers. These findings demonstrate that the use of AI for learning history can influence opinions through both intrinsic and intentional framing mechanisms, even when the content remains factually accurate. As AI becomes integral to information acquisition, recognizing pathways of influence based not only on user-manipulated content but also on models’ latent biases is essential for understanding AI’s broader societal impacts. Keywords: large language models, generative artificial intelligence, AI history, AI framing, AI persuasion Introduction such clearly prejudiced content. Rather, they can produce out­ The rapidly improving capability of large language models (LLMs) puts that appear neutral but convey subtle nuances or emphases to produce human-like text, images, and video has caused wide­ that potentially influence opinions (10). We examine this more spread concern about their potential to spread inaccurate infor­ elusive mechanism of AI persuasion, latent bias, and compare mation, conspiracy theories, and disinformation (1, 2). A growing its effect to the more commonly studied prompting bias, where users explicitly employ or instruct AI to convey a particular per­ body of research thus examines how exposure to LLMs can affect spective (2, 4–8). people’s views, ranging from beliefs in blatant falsehoods to Whereas prior work on prompting bias has focused on political opinions about legitimate policy debates (2). Much of this work and policy contexts (2, 5, 6, 8, 10) or outright misinformation (1, 4, has focused on content intentionally crafted by LLM users to per­ 7), our empirical application examines the use of LLMs to elicit suade, including propagandistic statements, personalized argu­ facts. Specifically, we consider historical events, since learning ments, and conversations with AI agents (2–8). about history informs attitudes and policy preferences (12), peo­ This research overlooks a less overt but potentially significant ple are increasingly using AI tools like ChatGPT and Google’s AI mechanism of persuasion: the underlying, latent biases in LLMs. Overview to obtain historical information (13),b and private com­ Biases can be introduced at multiple stages of the LLM training panies are using AI to produce alternative information sources, process, resulting in ideological leanings that can be difficult to such as xAI’s “Grokipedia.”c Our analyses test how both AI’s latent remove (9–11). In some cases, these biases are unintentional; and user-prompted framing of factually accurate historical con­ in other cases, developers can guide models to express particular tent can affect individuals’ opinions about social and political perspectives. For example, xAI’s Grok chatbot generated racist issues. and anti-antisemitic material on X after its instructions were al­ We begin our study by evaluating latent biases. We compare tered to permit “politically incorrect” claims.a To be sure, most default GPT-4o and Wikipedia summaries of two events that latent biases, whether intentional or unintentional, do not yield took place in the United States during the 20th century: a large Competing Interest: The authors declare no competing interests. Received: September 17, 2025. Accepted: December 10, 2025 © The Author(s) 2026. Published by Oxford University Press on behalf of National Academy of Sciences. This is an Open Access article dis­ tributed under the terms of the Creative Commons Attribution-NonCommercial License (https://creativecommons.org/licenses/by-nc/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited. For com­ mercial re-use, please contact reprints@oup.com for reprints and translation rights for reprints. All other permissions can be obtained through our RightsLink service via the Permissions link on the article page on our site—for further information please contact journals.permissions@oup.com. 2 | PNAS Nexus, 2026, Vol. 5, No. 3 Downloaded from academic.oup.com/pnasnexus/article/5/3/pgag022/8503065 by PricewaterhouseCoopers AG user on 24 October 2026 Fig. 1. Pairwise comparisons testing the effect of reading AI summaries of historical events on readers’ opinions. Statistical significance calculations incorporated an adjusted false discovery rate (FDR) based on the Benjamini–Hochberg procedure. labor strike and student protests calling for greater ethnic minor­ RQ3. How do the AI summaries’ effects vary based on the corres­ ity representation in academia. Following recent research on AI pondence between readers’ political ideology and the summaries’ persuasion (6), the default summaries are generated by instruct­ ideological framing? ing the model to take on the role of a relevant expert (i.e. a “histor­ ian and teacher”). To assess the relative influence of latent and Results prompting biases, we compare similar summaries that purpose­ We address RQ1 and RQ2 by comparing the opinions of readers of fully frame the events in politicized ways. This prompting bias is AI summaries to the opinions of those who read the Wikipedia introduced by following recent work that instructed LLMs to adopt summaries. Figure 1 shows that both the AI summaries with de­ a politicized perspective, or “role-play” (8). Namely, we use survey fault framing and the AI summaries with liberal framing led to data to seed the model with “opinions,” yielding summaries that more liberal opinions compared to the Wikipedia summaries portray the same historical events with either liberal or conserva­ (Default: mean difference = 0.1, Cohen’s d = 0.14, P < 0.05; Liberal: tive framing. The first two research questions are thus: mean difference = 0.21, Cohen’s d = 0.28, P < 0.001). Meanwhile, readers of AI summaries with conservative framing reported RQ1. What is the effect of default-framed AI summaries of more conservative opinions relative to Wikipedia (mean difference historical events on readers’ opinions, compared to Wikipedia = −0.11, Cohen’s d = −0.13, P < 0.05). Similar results were observed summaries? when analyzing each event separately. The effect sizes are similar in magnitude to those in prior re­ RQ2. What is the effect of inducing ideological framing in AI sum­ search (6). On the response scale, average opinions after reading maries of historical events on readers’ opinions, compared to each summary fell between 3 (moderate) and 4 (somewhat lib­ Wikipedia summaries? eral). The average opinion after reading either the default (3.57) or liberal AI text (3.67), compared with the average opinion after reading the Wikipedia text (3.47), represents a shift from a moder­ People filter information through their own ideologies and belief ate stance to leaning towards a somewhat liberal stance. The systems. Motivated reasoning theory suggests a tendency to favor average opinion after reading the conservative AI text (3.36) indi­ belief-confirming information (14), potentially leading to reinforce­ cates a slight shift towards more moderate opinions. ment from ideologically aligned frames. Meanwhile, encountering To examine whether readers’ political ideology moderated how counter-attitudinal views might provoke defensive reactions and politically framed AI summaries affected their opinions (RQ3), we backlash (15). Therefore, we additionally examine heterogeneous conditioned our preceding analysis on self-reported ideology. The effects of the AI summaries across readers’ ideologies. average marginal effects (AMEs) across reader ideology in Fig. 2 Shu et al. | 3 Downloaded from academic.oup.com/pnasnexus/article/5/3/pgag022/8503065 by PricewaterhouseCoopers AG user on 24 October 2026 Fig. 2. The AME of reading AI summaries of historical events, relative to reading Wikipedia summaries, conditional upon readers’ ideology. Lines denote 95% CIs. Calculations of statistical significance incorporate the Benjamini–Hochberg procedure. show that the default AI summaries did not affect readers’ opin­ opinions overall, although this effect was primarily driven by con­ ions differently from the Wikipedia summaries for liberal, moder­ servative respondents. ate, or conservative participants. In contrast, the liberal-framed This study is not without limitations. We focused on two events AI summaries led to more liberal opinions for participants across to ensure precise estimates, so further work is needed to examine all ideology groups (liberals: AME = 0.205, P < 0.05; moderates: whether the findings generalize to a broader range of events and AME = 0.177, P < 0.05; conservatives: AME = 0.190, P < 0.05). The historical periods, as well as other domains beyond history. The AI summaries with conservative framing resulted in more conser­ degree of latent bias may also vary across models, and future vative opinions, although this effect was statistically significant work should consider the impacts of different types of training (α = 0.05) only for those who already had a conservative outlook data and posttraining alignment techniques (11). More research (conservatives: AME = −0.173; P < 0.05). We conducted Bayesian is also needed to understand how the interaction of latent and analyses to evaluate the support for the null findings and found prompting biases shapes AI’s persuasiveness. For example, one moderate support for all null results and strong support for the explanation for the observed ideological asymmetry (Fig. 2) is null effect of the conservative summary on liberals (see Section that the liberal texts were more congruent with the latent G of the Supporting Information [SI] document). bias of the model, whereas the conservative prompt may have conflicted with the underlying representations used to generate a response (9). This study advances our understanding of the persuasive ef­ Discussion fects of AI and, more generally, its potential risks. Previous re­ Our results suggest that using popular AI tools to learn about search on AI persuasion has largely studied the influence of AI history can influence people’s opinions on social and political is­ material and interactions that are designed to persuade (2, 5, 6, sues. Default GPT-4o-generated summaries of historical events 8)—what we conceptualize as prompting bias. Complementing made readers’ opinions more liberal compared with Wikipedia this work, our findings identify the persuasiveness of AI’s latent summaries, demonstrating the persuasive effects of LLMs’ latent biases (10) and the implications of using AI to obtain factual infor­ biases. mation. The presence of latent biases means that simply using While the experiment tested only two events, supplementary chatbots to learn about the past can influence people’s opinions analyses showed that GPT-4o tended to generate liberal-framed at a magnitude comparable to prompting-biased AI material default summaries for a wide range of similar events, including (Fig. 1). Beyond AI’s persuasiveness, our study raises important anti-union protests and anti-Black race riots. This framing per­ questions about how the increasingly routine and widespread re­ sisted even when we used a simpler system message that did liance on AI tools to acquire and synthesize information and not instruct the model to take on the role of a relevant expert to knowledge (13) can shape our understanding of the social world. generate the summaries (see Section L the SI). This suggests that conservative framing in content generated by GPT-4o may require explicit prompting, whereas liberal framing could be the result of both latent and prompting bias. Materials and methods The main results also indicate that AI summaries offering fact­ We conducted a preregistered survey experiment approved by the ually accurate yet ideologically framed historical accounts led to Institutional Review Board at Yale University (#2000037333). All alignment between readers’ opinions the summaries’ ideological participants provided informed consent before joining the study. framing. Readers of the liberal summaries reported more liberal The experiment had 1,912 participants, sampled proportionally opinions, regardless of their own self-reported ideology. Readers to the United States population along several sociodemographic of the conservative summaries exhibited more conservative dimensions. 4 | PNAS Nexus, 2026, Vol. 5, No. 3 Each subject was randomly assigned to read summaries of review & editing), Keitaro Okura (Data curation, Investigation, two historical events, the Seattle General Strike (SGS) and the Methodology), and Thomas R. Davidson (Formal analysis, Third World Liberation Front (TWLF) student protests, taken Investigation, Methodology, Writing—original draft, Writing— from Wikipedia or generated by GPT-4o with either a default AI review & editing) framing, a liberal framing, or a conservative framing. The SGS was the first “general” strike in the United States, during which 65,000 workers in Seattle stopped work from 1919 February 6 to Data Availability 11. The TWLF protests involved university student groups in The data and coding script used to conduct the analyses in this 1968 that campaigned for curricula representing the histories study can be found at https://osf.io/zy9eu/overview. Downloaded from academic.oup.com/pnasnexus/article/5/3/pgag022/8503065 by PricewaterhouseCoopers AG user on 24 October 2026 and cultures of students of color, leading to the establishment of Ethnic Studies departments. The SI contains further discussion of these events and information on the construction of the sum­ maries. After reading each summary, participants reported their References opinions on related social and political issues, such as their views 1 Spitale G, Biller-Andorno N, Germani F. 2023. AI model GPT-3 on the appropriateness of strikes (relevant to the SGS) and the use (dis)informs us better than humans. Sci Adv. 9(26):eadh1850. of curricula to advance social justice causes (relevant to the TWLF). 2 Salvi F, Ribeiro MH, Gallotti R, West R. 2025. On the conversation­ Participants’ answers were scored on a five-point scale (1 = ex­ al persuasiveness of GPT-4. Nat Hum Behav. 9(8):1645–1653. tremely conservative; 5 = extremely liberal), and the mean across 3 Costello TH, Pennycook G, Rand DG. 2024. Durably reducing con­ both events forms our outcome measure. spiracy beliefs through dialogues with AI. Science. 385:eadq1814. To address RQ1 and RQ2, we conduct pairwise comparisons 4 Goldstein JA, Chao J, Grossman S, Stamos A, Tomz M. 2024. between each treatment. The results were adjusted using the How persuasive is AI-generated propaganda? PNAS Nexus. 3(2): Benjamini–Hochberg procedure to account for multiple hypoth­ pgae034. esis tests. For RQ3, we use an ordinary least squares regression 5 Hackenburg K, Margetts H. 2024. Evaluating the persuasive influ­ model and the Lin estimator to predict average opinion with an ence of political microtargeting with large language models. Proc interaction term comprising the type of summaries and reader Natl Acad Sci U S A. 121(24):e2403116121. ideology. Further methodological details are provided in the SI. 6 Argyle LP, et al. 2025. Testing theories of political persuasion us­ ing AI. Proc Natl Acad Sci U S A. 122(18):e2412815122. Notes 7 Dash S, Xu Y, Jalbert M, Spiro ES. 2025. The persuasive potential of AI-paraphrased information at scale. PNAS Nexus. 4(7): a https://www.npr.org/2025/07/09/nx-s1-5462609/grok-elonmusk- pgaf207. antisemitic-racist-content. Accessed July 2025. 8 Hackenburg K, Ibrahim L, Tappin BM, Tsakiris M. Comparing the b https://www.nytimes.com/2025/06/16/magazine/ai-history-his persuasiveness of roleplaying large language models and human experts torians-scholarship.html. Accessed December 2025. on polarized U.S. political issues. AI & Society, 2025. c https://www.nytimes.com/2025/10/27/technology/grokipedialaunch- 9 Rozado D. 2024. The political preferences of LLMs. PLoS One. 19(7): elon-musk.html Accessed October 2025. e0306621. 10 Potter Y, Lai S, Kim J, Evans J, Song D. Hidden persuaders: LLMs’ Acknowledgments political leaning and their influence on voters. In: Al-Onaizan Y, The authors thank the editor and anonymous reviewers for their Bansal M, Chen Y-N, editors. Proceedings of the 2024 Conference on valuable suggestions. The authors also thank Bart Bonikowski Empirical Methods in Natural Language Processing. Association for and the participants of New York University Department of Computational Linguistics, Miami, Florida, USA, November Sociology’s colloquium for their helpful comments and feedback. 2024. p. 4244–4275. 11 Bai X, Wang A, Sucholutsky I, Griffiths TL. 2025. Explicitly un­ biased large language models still form biased associations. Supplementary Material Proc Natl Acad Sci U S A. 122(8):e2416228122. Supplementary material is available at PNAS Nexus online. 12 Ginsberg B, Bachner J. Warping time: how contending political forces manipulate the past, present, and future. University of Michigan Press, Ann Arbor, 2023. Funding 13 Karell D, Shu M, Davidson T, Okura K. 2025. Generating the past: This research was not supported by any external funding. how artificial intelligence summaries of historical events affect knowledge. Soc Sci Comput Rev. https://doi.org/10.1177/ 0894439325140974. Author Contributions 14 Taber CS, Lodge M. 2006. Motivated skepticism in the evaluation Matthew Shu (Conceptualization, Data curation, Formal analysis, of political beliefs. Am J Pol Sci. 50(3):755–769. Investigation, Methodology, Writing—original draft), Daniel 15 Bail CA, et al. 2018. Exposure to opposing views on social media Karell (Conceptualization, Formal analysis, Funding acquisition, can increase political polarization. Proc Natl Acad Sci U S A. Investigation, Methodology, Writing—original draft, Writing— 115(37):9216–9221.