Search on EES

Where next in the quest for better evidence — after this year’s picks for the Nobel Prize in Economics 

The 2021 Nobel Prize in Economics was awarded jointly to David Card, Joshua D. Angrist and Guido W. Imbens.  Angrist and Imbens were so awarded because “for their methodological contributions to the analysis of causal relationships” – a subject close to the heart of modern evaluation. Indeed, the methods pioneered by Angrist and Imbens have clearly raised the bar for how evaluators weigh evidence and plan studies. Evidence-based evaluation owes much to the rigor introduced by the new Nobel Laureates over the course of their celebrated careers.  

This blog looks to the future of the evaluation discipline as it builds on the work of these and other innovators.  The blog authors use 3ie’s database of evaluations to illustrate the broadening base of support for the experimental and quasi-experimental methods proposed by Angrist and Imbens.  It then outlines the challenges of integrating these insights into policy.  With multiple competing suggestions utilising evidence-based frameworks, adoption is too often sporadic and temporary.   

How can funders achieve systems change?  

Much of evaluation focuses on suggestions to donors, policy-makers and other relevant organisations.  These parties are increasingly turning their attention to systems change as their guiding ethos.  Analysis of systems is nothing new to evaluation in principle and the discipline has evolved techniques and frameworks to tackle the problem, such as the Systems Evaluation Theory (SET).  The new trend is evidently rooted in frustration with the patchy medium and long term results of ad hoc interventions that might, in isolation, be justified and evaluated positively.  This blog provides a useful overview of systems theory in practice as it stands at the end of 2021 and leans into the often-ignored attitudinal change required to fulfil the potential of the approach.  Although the blog could have benefited from a donor-centric perspective on why this area of evaluation is receiving more attention, it is still worth reading as a primer on this emerging topic.  

What do we need for better monitoring?  

Operated by UNICEF and Better Evaluation, the Global Partnership for Better Monitoring aims, as the name would suggest, to improve the monitoring component of M&E.  Too often dismissed as merely routine or even conducted long after the event, few would deny that monitoring is in need of improvement and the Partnership’s aims to tackle this with ideas including a framework for knowledge sharing and inter-organisational cooperation.  This could provide monitoring with the same professionalisation and investment as evaluation now enjoys.  The blog summarises the inputs phase of the project, during which the M&E community provided its views on the challenges and opportunities for monitoring going forward. Among the insights generated are that monitoring is increasingly blurred with evaluation, that there is a clear need for more iterative theories of change and that management must evolve in step with monitoring if it is to have any chance of succeeding. 

Are you considering social and environmental criteria in your evaluations?  

This blog, and many outreach efforts like it, are attempting to gauge the adoption of new environmental considerations in mainstream evaluation activity.  This short piece and the others like it can be considered companion pieces to the mammoth UN ‘Stock-Taking Exercise on Policies and Guidance of UN Agencies in Support of Evaluation of Social and Environmental Considerations’ published earlier this year.  The take-away is that tentative efforts to establish system-wide M&E guidance on environmental and social matters have progressed, albeit gradually in most quarters.  Significantly, much of the progress is occurring not because of substantive central guidance but because of spontaneous reactions to perceived need.  In other words, although good intentions abound, current efforts may ultimately be undermined by their lack of coordination.  Such gripes – that UN agencies and other concerned parties are proceeding in a good but disorganised manner – are of course perennial complaints.  The stock-taking report at least acknowledges that part of the impetus to autonomous action is perceived inertia at the centre. UNEG is itself inhibited by the voluntary nature of most contributions but it has collected valuable insights on the limited guidance provided to practitioners, who too often find themselves without the frameworks required to conduct socially and environmentally conscious evaluations. 

Development Agenda for Egypt: Fostering Government-Research Partnerships 

Interesting though this blog’s notes on J-PAL’s relationship with the ambitious development goals of the Egyptian government are, the real value of this piece is its succinct summary of how evidence-based research can influence policy.  As discussed earlier, this – rather than the academic substance of such research – is increasingly the stumbling block for M&E projects.  Using the example of Indonesia, the blog demonstrates how sufficiently rigorous and well-presented studies can direct policy consistently. In exchange for reliable support and the freedom to operate their research projects, Indonesia gained “immediately useful and contextually appropriate guidance on how to adapt and scale programs to improve outcomes and use scarce resources well.”  The blog outlines how a similar network of institutional partnerships in Egypt is preparing the ground for an effective series of studies designed to meet Egypt’s evolving needs with the highest standard of evidence-based research.  J-PAL’s future success, or lack thereof, in this endeavour may tell us much about the generalisability of its experience in Indonesia, especially as the political environment is decidedly more challenging.