How can we leverage LLM chatbots to support policymakers’ in understanding and using evidence?
Project Summary
Although the last few decades have seen rapidly increasing interest in evidence-based policymaking and a growing evidence base on “what works,” adoption and use of evidence-based solutions remains low. Myriad informational barriers may impede policymakers’ ability to understand, translate, and use evidence in their own contexts. For instance, they need to know not only whether evidence exists and where to find it, but also how to understand and evaluate its usefulness. In this project, The People Lab is leveraging large language models (LLMs) to summarize scientific research and develop chatbots that can assist policymakers in synthesizing and translating evidence, and coach them through assessing its relevance, strength, and applicability.
Why is this issue important?
Despite decades of investment in rigorous social science, much of what we know about “what works” fails to reach and impact the decision makers who shape public policy. Evidence is fragmented across academic papers, clearinghouses, and think tanks—leaving policymakers without the time, tools, or training to find, interpret, and apply it. The result is a persistent gap between the production of evidence and its practical use in guiding real-world decisions, from workforce development and public health to housing and climate adaptation. However, LLMs offer new opportunities to support policymakers in synthesizing, translating, and interpreting available evidence.
What are we doing?
We are using LLMs to extract and summarize important dimensions of scientific evidence (effect sizes, statistical significance, political context, etc.) from published papers and reports across a wide range of policy areas. We will use this data to train and develop an AI-driven “PolicyBot.” This interactive, domain-specific tool will synthesize evidence and support policymakers in using it effectively by serving both as (a) an assistant, providing tailored answers to policy questions drawn from curated evidence bases; and (b) a coach, guiding users through the strengths, weaknesses, and applicability of the evidence so they can better weigh tradeoffs and build trust in the underlying evidence base. We will then conduct a series of studies to examine how policymakers interact with and use PolicyBot, and to test whether the tool influences trust in or demand for evidence.
What have we learned?
The project is currently in the design phase. We expect to launch studies of “PolicyBot” in 2026.