On the Evidence

Latest Episodes:

100 | Robert Shea on the Federal Government’s Progress in Using Evidence Aug 02, 2023

Show notes

In May 2023, Mathematica hosted a convening on Capitol Hill about embedding evidence in federal decision making, with a focus on the legacy of the nearly five-year-old Foundations for Evidence-Based Policymaking Act, also known as the Evidence Act. One of the attendees that day was Robert Shea, an expert on performance improvement in government who served on the bipartisan U.S. Commission on Evidence-Based Policymaking, a group whose recommendations informed the Evidence Act. Shea is the chief executive officer for GovNavigators, a government management consulting firm, where he hosts a podcast called The GovNavigators Show. His career has also included posts at the White House Office of Management and Budget, the Senate Committee on Governmental Affairs, and the House Committee on Government Reform and Oversight. On this episode of Mathematica’s On the Evidence podcast, Shea discusses the improvements he has witnessed since the mid-1990s in the use of data and evidence in federal decision making. “We have seen a sea change in the ability of agencies to understand that they need to articulate what they’re trying to accomplish in terms of outcomes, report that transparently,” Shea says on the episode. “A lot of what we’re talking about today is sort of dig deeper—find out whether it’s what we’re doing that’s contributing to the ultimate outcome rather than some other factor.” This episode is part of an occasional series on the show called Evidence in Government, which explores new developments in the halls of government and the role that evidence can play in decisions that could improve people’s lives. Mike Burns, Mathematica’s senior director of communications and public affairs at Mathematica, conducts the Evidence in Government interviews. SHOW NOTES Listen to The GovNavigators Show, a podcast Shea hosts with Adam Hughes about government management: https://www.govnavigators.com/podcast Watch a recording of the event Mathematica hosted in May 2023 on the Evidence Act: https://www.mathematica.org/events/evidence-at-work-for-the-american-people Learn more about the U.S. Commission on Evidence-Based Policymaking, which made recommendations ahead of the Foundations for Evidence-Based Policymaking Act of 2018: https://obamawhitehouse.archives.gov/omb/management/commission_evidence Read a fact sheet from the Bipartisan Policy Center on the Foundations for Evidence-Based Policymaking Act: https://bipartisanpolicy.org/blog/fact-sheet-foundations-for-evidence-based-policymaking-act/ Read a press release about the resolution to establish a new commission on evidence-based policymaking: https://kilmer.house.gov/media/press-releases/kilmer-spearheads-legislative-change-with-introduction-of-evidence-based-policymaking-resolution Read the bipartisan resolution to establish a new commission on evidence-based policymaking to further embed the use of data and evidence in federal decision making: https://www.congress.gov/bill/118th-congress/house-concurrent-resolution/49/text?s=1&r=1&q=%7B%22search%22%3A%5B%22Evidence+commission%22%5D%7D

99 | Understanding Why More Eligible Kids Got Free Meals During Pandemic Summers Jul 20, 2023

Show notes

Leading up to the COVID-19 pandemic, only about one in every seven students served by the National School Lunch Program also participated in free summer meals programs provided by the federal Food and Nutrition Service. The low ratio of students accessing meals in the summer compared to the school year has been dubbed the “summer food gap.” Policymakers and anti-hunger advocates have long worried that millions of children from households with low incomes aren’t getting enough to eat when school is out of session, posing risks to children’s health, learning, and overall well-being. This episode of On the Evidence examines what happened during the COVID-19 pandemic when the federal government temporarily waived some of its program rules, giving meal providers much more flexibility in how they fed students during the school year and in the summer months. With the waivers in place, program operators could prioritize convenience for families and avoid the spread of infection. For example, parents or guardians could pick up meals for their children at the curb or a drive-through; they could grab more than one meal at a time; and they could pick up the meals outside of traditional service hours, such as in the early mornings or evenings. The waivers also cleared the way for more summer meals sites to open, which helped establish new sites closer to where some families lived. Recent research from the Food and Nutrition Service and Mathematica shows that in 2020, with temporary program waivers in place because of the pandemic, these federal summer meals programs reached many more children and served many more meals than usual. The podcast examines why access to meals improved in the summer of 2020 and what it could mean for summer meal access now that COVID-19 is no longer a public health emergency, and those temporary waivers have ended. The episode features three guests: Shana Christrup, Lindsay Aguilar, and Veronica Severn. Christrup is the public health director for the Bipartisan Policy Center, which released a report in January 2022 that recommended changes to federal child nutrition programs, including changes that would expand access to meals in the summer. Aguilar is the food services director for the Tucson Unified School District. Severn is a survey researcher at Mathematica who coauthored the recent report on the pandemic-era waivers for child nutrition programs in the spring and summer of 2020. Read the report from the Food and Nutrition Service and Mathematica on child nutrition program operations from March through September 2020: https://smo-study.mathematica.org/ Read the report from the Bipartisan Policy Center on strengthening child nutrition programs: https://bipartisanpolicy.org/report/child-nutrition-programs/ Read Route Fifty’s story on lessons from the pandemic about feeding more children during the summer: https://www.route-fifty.com/health-human-services/2023/06/federal-covid-changes-helped-feed-twice-many-kids-during-summer/387158/

98 | Developing Education Programs to Prevent Violence and Crime in Latin America and the Caribbean Jul 06, 2023

Show notes

This episode of Mathematica’s On the Evidence podcast focuses on how research informs funding strategies to tackle an urgent social challenge. A couple of years ago, Mathematica reviewed evidence from across the world on whether education programs can prevent or mitigate violence and crime. USAID commissioned the report to help develop effective strategies for leveraging the education sector to address violence and crime in Latin America and the Caribbean. Although Mathematica’s researchers identified almost two dozen promising programs in the review, they also noted the need for further research. For example, few of the studies show causal impacts or provide evidence from an effective program in Latin America or the Caribbean. To build on findings from Mathematica’s evidence review, USAID and the International Rescue Committee(IRC) partnered to launch an initiative called USAID Research in Education for Transformative Opportunities (RETO), which provides tailored evidence to decision makers in governmental and non-governmental institutions, such as local ministries of education. The initiative seeks to help those education leaders implement effective programs and policies in Northern Central America that will reduce local violence and crime. The guests for this episode are Juan Carlos Rodríguez, Katie Appel, Melissa Chiappetta, and Emilie Bagby. Rodríguez and Appel are the director and deputy director, respectively, for USAID's RETO activity implemented by the IRC and its 12 national partners. Chiappetta is a senior education advisor with the Office of Regional Sustainable Development within USAID's Bureau for Latin America and the Caribbean. Bagby is a director at Mathematica, where she oversees our international education research. Find the full transcript here: mathematica.org/blogs/developing-education-programs-to-prevent-violence-and-crime-in-latin-america-and-the-caribbean Read the evidence review by Mathematica for USAID on the effect of education programs on violence, crime, and related outcomes: https://pdf.usaid.gov/pdf_docs/PA00XGXT.pdf Learn more about RETO: https://www.youthpower.org/usaid-research-education-transformative-opportunities-reto Learn more about IRC’s Airbel Impact Lab: https://airbel.rescue.org/?_ga=2.189623531.1744177215.1687884026-800337516.1664199528 Learn more about Mathematica’s evaluations for USAID’s Latin America and the Caribbean Reads (LAC Reads) initiative, which included the evidence review on education programs to prevent or mitigate violence and crime: https://staginginter.mathematica.net/projects/latin-america-and-the-caribbean-reading-evaluation

97 | Building on the Evidence Act Jun 21, 2023

Show notes

In late May 2023, Mathematica coordinated with Congressman Derek Kilmer’s office to host a discussion on Capitol Hill about the impact of the Foundations for Evidence-Based Policymaking Act (Evidence Act) and how to further embed the use of data and evidence in federal decision making. This episode of On the Evidence features edited audio from the event with remarks from Kilmer and his House colleague, Rep. William Timmons, in addition to a panel discussion with experts from government, think tanks, research organizations, and academia. Kilmer represents Washington State’s Sixth District and was one of the Evidence Act’s original co-sponsors. He is also the sponsor of a new bipartisan resolution to establish a new commission on evidence-based policymaking that would advise Congress on how to make greater strides in embedding the use of data and evidence in federal agencies but also in Congress itself. Timmons represents South Carolina’s Fourth District and is a co-sponsor of the resolution to establish a new evidence commission. The other speakers included Robert Groves, Nick Hart, Ruth Neild, and Susan Jenkins. Groves is a provost of Georgetown University and served as a member of the previous U.S. Commission on Evidence-Based Policymaking, whose recommendations informed the Evidence Act. Hart is the president and CEO of Data Foundation and served as policy and research director for the U.S. Commission on Evidence-Based Policymaking. Neild is an executive director of education and employment research at Mathematica and previously served as delegated director of the Institute of Education Sciences within the U.S. Department of Education. Jenkins directs the Division of Evidence, Evaluation and Data Policy at the U.S. Department of Health and Human Services, where she also serves as an evaluation officer. A full transcript of the episode is available at mathematica.org/blogs/building-on-the-evidence-act-to-increase-the-positive-impact-of-federal-policymaking Learn more about the U.S. Commission on Evidence-Based Policymaking, which made recommendations ahead of the Foundations for Evidence-Based Policymaking Act of 2018: https://obamawhitehouse.archives.gov/omb/management/commission_evidence Read a fact sheet from the Bipartisan Policy Center on the Foundations for Evidence-Based Policymaking Act: https://bipartisanpolicy.org/blog/fact-sheet-foundations-for-evidence-based-policymaking-act/ Read a press release about the resolution to establish a new commission on evidence-based policymaking: https://kilmer.house.gov/media/press-releases/kilmer-spearheads-legislative-change-with-introduction-of-evidence-based-policymaking-resolution Read the bipartisan resolution to establish a new commission on evidence-based policymaking to further embed the use of data and evidence in federal decision making: https://www.congress.gov/bill/118th-congress/house-concurrent-resolution/49/text?s=1&r=1&q=%7B%22search%22%3A%5B%22Evidence+commission%22%5D%7D

96 | Addressing Fragmented Health Care Jun 07, 2023

Show notes

The fragmentation of outpatient health care drives up the cost of care and worsens the quality of care that patients receive, posing a risk to patients’ health. On this episode of Mathematica’s On the Evidence podcast, guests James Lee of the Center for Medicare & Medicaid Innovation, Knitasha Washington of ATW Health Solutions, Bob Phillips of the Center for Professionalism and Value in Health Care, and Lori Timmins of Mathematica discuss recent research on the nature of the problem and federal initiatives that have sought to address it. A full transcript of the episode is available at https://www.mathematica.org/blogs/understanding-and-addressing-fragmented-outpatient-health-care Read a press release synthesizing key takeaways from a series of peer-reviewed journal articles on fragmented outpatient care based on studies conducted by Mathematica with support from the Centers for Medicare & Medicaid Services: https://www.mathematica.org/news/new-studies-reveal-that-fragmented-care-persists-despite-efforts-to-improve-primary-care-and-care Learn more about how the Innovation Center and Mathematica are advancing understanding of primary care through an evaluation of the Comprehensive Primary Care Plus model: https://www.mathematica.org/projects/evaluating-the-nations-largest-primary-care-delivery-initiative Learn more about an ongoing evaluation by Mathematica for the Center for Medicare & Medicaid Innovation to determine whether medical practices’ participation in the Primary Care First alternative payment model improves quality and reduces costs for Medicare fee-for-service beneficiaries: https://www.mathematica.org/projects/evaluation-of-the-primary-care-first-model Read the 2021 report from the National Academies of Sciences, Engineering, and Medicine on implementing high quality primary care: https://nap.nationalacademies.org/catalog/25983/implementing-high-quality-primary-care-rebuilding-the-foundation-of-health

95 | The Role of Evidence in Solutions Journalism May 25, 2023

Show notes

The 95th episode of Mathematica’s On the Evidence podcast features author Tina Rosenberg discussing the role of data and other evidence in supporting solutions journalism. In 2013, Rosenberg co-founded the Solutions Journalism Network, which challenges journalists to show whether a solution they are covering is effective using available data or qualitative results. The episode explores the nature of solutions journalism and how researchers who evaluate policies and programs can contribute to evidence-based reporting about solutions. A video recording and related transcript from the LinkedIn Live event on May 2, 2023 is available at mathematica.org/blogs/tina-rosenberg-on-the-role-of-evidence-in-solutions-journalism Read the last installment of the Fixes column that Rosenberg and David Bornstein co-wrote for 11 years at The New York Times: https://www.nytimes.com/2021/11/11/opinion/fixes-solutions-journalism-lessons.html Learn how to implement the principles of solutions journalism in your work by taking free online courses offered by the Solutions Journalism Network’s Learning Lab: https://learninglab.solutionsjournalism.org/ Explore the Solutions Journalism Network’s Story Tracker, a curated database of rigorous reporting on responses to social problems: https://www.solutionsjournalism.org/storytracker Read about the impact of solutions stories through the Solutions Journalism Network’s Impact Tracker: https://www.solutionsjournalism.org/impact Read a solutions story by Rosenberg about how to triple voter turnout: https://www.nytimes.com/2020/09/19/opinion/voter-turnout-2020-election.html Read a solutions story by Michael Kimmelman about housing people who were living on the streets of Houston: https://www.nytimes.com/2022/06/14/headway/houston-homeless-people.html

94 | Christina Ciocca Eller on White House Year of Evidence for Action May 10, 2023

Show notes

Episode 94 of On the Evidence is the second installment in a new occasional series on the show called Evidence in Government, where guests talk about new developments in the halls of government and the role that evidence can or should play in decisions that could improve people’s lives. On this episode, Christina Ciocca Eller talks about a recent push at the White House to accelerate the comingling of research and public policy to improve the lives of the American people. Ciocca Eller is an assistant professor of sociology and social studies at Harvard University who previously served as the assistant director for evidence and policy in the White House Office of Science and Technology Policy under the Biden-Harris Administration. In that role, she co-led the White House’s Year of Evidence for Action initiative. Following the one-year anniversary of the Year of Evidence for Action, Ciocca Eller and Mathematica’s Mike Burns discuss what motivated the White House’s intensified focus on data, evidence, and science in the federal government; what the effort achieved; and what’s next in the movement to embed evidence in decision making. A full transcript of the episode is available at mathematica.org/blogs/former-biden-harris-administration-official-christina-ciocca-eller-on-turning-evidence-into-action

93 | Inside Arizona’s Investment in Evidence-Based Foster Care Prevention Apr 26, 2023

Show notes

On this episode, we discuss the Family First Prevention Services Act in the context of one state’s efforts to build evidence for a parent education program thought to prevent child abuse and neglect. Katherine Guffey, Meg Dygert, and Allon Kalisher discuss a parent education program in Arizona that Mathematica evaluated, the Family First law, and the long-term implications of the law’s provisions around prevention services and evidence of effectiveness. Guffey is the executive consultant to the director of the Arizona Department of Child Safety. Dygert is the senior policy associate for child and family well-being at the American Public Human Services Association. Kalisher is a senior researcher at Mathematica. Find a full transcript of the episode at https://www.mathematica.org/blogs/inside-arizonas-efforts-to-invest-in-evidence-based-foster-care-prevention-services. Read Mathematica’s evaluation of Arizona’s the Nurturing Skills for Families program: https://mathematica.org/publications/impact-evaluation-of-the-nurturing-parenting-program-nurturing-skills-for-families Read a policy brief from the American Public Human Services Association on unlocking the “prevention services” in the Family First Prevention Services Act: https://files.constantcontact.com/391325ca001/09019575-0805-4650-b3c5-4a6c58769a38.pdf Learn more about our live podcast with Tina Rosenberg about the role of evidence in solutions journalism on May 2, 2023 at 2 p.m. ET: https://www.linkedin.com/video/event/urn:li:ugcPost:7049466439790956544/ To submit comments or questions in advance of the LinkedIn Live event, email info@mathematica-mpr.com or call Mathematica's On the Evidence podcast to leave a voicemail at (609) 945-6600.

92 | Evidence-Based Policy in President Biden's 2024 Budget Apr 12, 2023

Show notes

In the president’s proposed budget for the 2024 fiscal year, the Biden White House outlines a spending plan that would expand health care insurance coverage for more Americans, extend free school meals to more children, and provide financial assistance for child care to more families. The budget includes more than a dozen references to evidence and calls for taking evidence-based or evidence-informed approaches to criminal justice, foster care prevention services, and registered apprenticeship programs. To understand what the president’s budget might mean for the future of evidence-based policymaking, Mathematica’s Mike Burns spoke with Caitlin Emma and Nichole Dunn for the latest episode of Mathematica’s On the Evidence podcast. This episode is part of a new occasional series for the On the Evidence that explores new developments in the halls of government and the role that evidence could play in informing decisions that affect people’s well-being. Find a full transcript of the episode at mathematica.org/ontheevidence. Learn more about our live podcast with Tina Rosenberg about the role of evidence in solutions journalism on May 2, 2023 at 2 p.m. ET: https://www.linkedin.com/events/7049466437156962304/comments/ Submit questions for Rosenberg in advance of the event by emailing info@mathematica-mpr.com. Read Emma’s coverage of President Biden’s budget as a re-election campaign document (https://www.politico.com/news/2023/03/09/biden-budget-reelection-campaign-00086452) and a challenge to House Republicans who have called for spending cuts (https://www.politico.com/news/2023/03/09/biden-budget-medicare-house-republicans-00086124). Read a summary of key provisions in the president’s budget that support the use of evaluation, data, and evidence to make government more effective: https://www.evaluation.gov/assets/resources/2024 Evidence Chapter in Brief.pdf Explore the American Rescue Plan Data and Evidence Dashboard, developed by Results for America and Mathematica. It shows how state, local, and tribal governments are spending $350 billion in fiscal recovery funds from the American Rescue Plan Act: https://results4america.org/tools/arp-dashboard/ Explore the Results for America's Economic Mobility Catalog, which helps local leaders identify and implement evidence-based strategies to improve economic mobility for their residents: https://results4america.org/page/economic-mobility-catalog/ Learn more about the What Works Nonprofit Fellowship program and the Local Government Fellowship program offered by Results for America: https://results4america.org/our-work/

91 | Solutions for Preventing Heart Attacks, Improving Patient Health, and Reducing Health Spending Mar 30, 2023

Show notes

Every year, Mathematica publishes dozens of new papers and reports, some of which surface fresh insights about how public agencies and private organizations can be more effective at improving public well-being. For the 91st episode of Mathematica’s On the Evidence podcast, we’re launching an occasional series focused on examples of solutions that recent research suggests are effective. All three solutions in this episode involve financial incentives that seek to reduce health care costs while improving the quality of care patients receive. One seeks to reduce the incidence of heart attacks, strokes, or other events related to cardiovascular disease. Another is focused on bringing down the cost of health care and improving patient health by providing primary care services at home. And the last one, again, seeks to keep costs down while improving patient health, but this time by using a unique payment model that provides incentives to hospitals to change the way they do business. Email jwogan@mathematica-mpr.com to provide feedback about our approach to covering solutions on the podcast. A full transcript of this episode is available here: https://mathematica-mpr.com/blogs/solutions-for-preventing-heart-attacks-improving-patient-health-and-reducing-health-spending Read the fourth annual report from Mathematica for the Centers for Medicare & Medicaid Services on the evaluation of the Million Hearts Cardiovascular Disease Risk Reduction Model: https://www.mathematica.org/publications/evaluation-of-the-million-hearts-cardiovascular-disease-risk-reduction-model-fourth-annual-report Learn more about the ongoing evaluation of the Million Hearts Cardiovascular Disease Risk Reduction Model: https://www.mathematica.org/projects/million-hearts-cardiovascular-disease-risk-reduction-model Read the seventh annual report from Mathematica for Center for Medicare & Medicaid Innovation on the evaluation of the Independence at Home Demonstration: https://www.mathematica.org/publications/evaluation-of-the-independence-at-home-demonstration-an-examination-of-year-7-the-first-year Learn more about the ongoing evaluation of the Independence at Home Demonstration: https://www.mathematica.org/projects/evaluation-of-the-independence-at-home-demonstration Read the report from Mathematica for the Center for Medicare & Medicaid Innovation on impacts from the first three years of the Maryland Total Cost of Care Model: https://www.mathematica.org/publications/evaluation-of-the-maryland-total-cost-of-care-model-quantitative-only-report-for-the-models-first Learn more about Mathematica’s ongoing evaluation of the Maryland Total Cost of Care Model for the Center for Medicare & Medicaid Innovation: https://www.mathematica.org/projects/evaluating-accountability-for-statewide-health-cost-and-quality-outcomes-cpc

Our TOPPODCAST Picks

Follow Us

Stay Connected

Related Podcasts

Links

Stay Connected