Making data reusable: lessons learned from replications of impact evaluations
In recent years, efforts to replicate the findings in scientific studies indicate that many results cannot be verified. In other words, reported findings cannot be reproduced using the original dataset and analysis code. The ‘replication crisis’ (as it has come to be known) appears to be a cross-disciplinary challenge. While this has led to a call for more replications, in practice there are few incentives for doing so. In the international development sector, there is an emphasis on developing interventions and policies that are grounded in rigorous evidence. Given the limited resoures available to tackle large scale challenges, it is imperative to ensure policymaking and programming draw upon lessons learned from evaluations of development interventions. But, given the replication crisis, how reliable is this evidence?
In its role as a producer and synthesizer of evidence, the International Initiative for Impact Evaluation (3ie) funds impact evaluations of development interventions and policies in low- and middle-income countries. In 2018, we embarked on a project to replicate published evaluation results using the data and analysis code submitted by evaluation teams. The talk will present the findings from this effort and discuss lessons learned and possible recommendations for various actors, hopefully with active participation from the audience.
Marie Gaarder provides general leadership, strategic direction and guidance to 3ie’s work in evaluation, synthesis, innovation and country engagement, in addition to overseeing the evaluation and synthesis office. Marie has over 19 years of experience managing operational and research projects with a development focus. Marie is the co-chair of the International Development Coordinating Group within the Campbell Collaboration, a member of the Research Ethics Review Committee of the Partnership for Economic Policy, and a member of the DFID-CDC Evaluation & Learning Programme Steering Group. Marie holds a PhD in Economics from University College London, an MSc in Economics from London School of Economics and a graduate degree in Political Science, Arabic and Economics from University of Oslo, Norway.
Upcoming lectures
30th October | Turning ‘evidence for development’ on its head | Ruth Stewart (Africa Centre for Evidence)
4th December | How the Global Innovation Fund uses impact forecasts to guide investment decisions | Ken Chomitz (Global Innovation Fund)
11th December | Using evidence in humanitarian decision-making | Sheree Bennett (IRC)
22nd January | Meta-ethnography to build middle-range theories: an exploration in three case studies | Audrey Prost (UCL)
19th February | Issues in the design and practice of randomised trials in implementation Science: the case of universal testing and treatment for HIV/AIDS | James Hargreaves (LSHTM)
4th March | Measuring the ‘hard to measure’ in development: abstract, multi-dimensional concepts and processes | Anne Buffardi (ODI)
11th March | Trials and tribulations of collecting evidence on effectiveness in disability inclusive development | Hannah Kuper (LSHTM)
1st April | Using causal-process-tracing middle-level theory to improve local predictions and planning for more effective interventions | Nancy Cartwright (Uni of Durham and UC San Diego)
22nd April | Emerging Lessons from Africa on Evidence Use for Policy and Implementation | Ian Goldman (Witwatersrand)
Admission
Contact
