TWG 6 – Methods and Designs

Search on EES

TWG Leader

TWG Leader

Sven Harten

Presentation of TWG

Evaluations often deal with multiple and multi-dimensional evaluation questions and assess complex evaluation objects in changing contexts. This requires evaluators to apply different methods and to analyze different types of data. Additionally, the evaluation community is multi-disciplinary relying on a wide set of analytical procedures from the social sciences, natural sciences, applied sciences and humanities. The toolbox of available designs and methods is thus rich and diverse.

This EES Thematic Working Group deals with issues around evaluation methods and designs. Specifically, since the evaluation community agrees that there is a potential value added from integrating different methods in evaluations, the primary focus for the first four years (2019-2023) will be on how to apply and integrate multiple methods in mixed/multi-method designs. This concept note is therefore written with a focus on method integration in mind. The members of the TWG will decide in 2019 on a bi-annual work program (until 2021) and at the end of the second bi-annual (2021-2023) will decide whether to maintain or change the thematic focus of the TWG.

RELEVANCE
The relevance of setting up this working group builds upon the following observations:
1. No single method is able to cope with interventions that affect multiple levels of analysis, involve short- and longterm outcomes, and address complex systems with interacting components. The TWG supports the evaluation community in selecting and integrating suitable quantitative and qualitative methods.
2. Evaluation can benefit from rigorous causal inference. The TWG supports the identification of causal effects and causal mechanisms through integrated cross- and within-case designs.
3. There is potential in the increasing availability of mixed- and multi-method approaches to evaluation. The TWG contributes to clear evaluation designs by providing formal protocols.
4. Evaluations increasingly draw on new types of data (including big data such as geo-spatial imagery, transactional data, mobile phone records or internet/app data) and use of innovative analytical tools like machine-learning. The TWG promotes the integration of emerging data types and analytical techniques in complex evaluations.
The TWG sees an increased need for methodological exchange and learning on method integration.

VISION
With this Thematic Working Group we strive to improve the quality of evaluations and strengthen evidence-based decision making through better methods and designs. Specifically, our aim for the coming four years is advancing the systematic integration of different methods in evaluations in order to generate more valid and reliable evidence.

 

Objectives of TWG

MISSION STATEMENT
The TWG “Methods and Designs” will be a collaborative space to share experiences, good practices and lessons learned round evaluation methods and designs while at least initially a particular focus will lie on systematically integrating different methods in evaluations. It builds on ongoing debates in mixed- and multi-method research within the evaluation discipline and in the broader social sciences. The TWG aims at bringing together evaluators and academics to exchange ideas and experience on innovative ways of combining methods and to develop systematic approaches of method integration in evaluation.

The main objectives of the TWG are to:

  • Facilitate methodological debate and exchange among EES members, specifically on the matter of method integration.
  • Bundle expertise and best practices in mixed-methods and multi-method approaches for evaluation.
  • Reconcile different methodological cultures in the evaluation community.
  • Transfer and adapt recent methodological innovations from the social sciences for the field of evaluation.
  • Work and advance towards a more systematic integration of multiple methods in evaluation.
  • Increase the availability of systematic approaches to method integration in evaluation.
  • Provide guidance for evaluators on the selection of appropriate methods and their integration.
  • Improve the integration of new types of data and data analysis in evaluation designs.

Accordingly the TWG helps bring together evaluators and academics interested in improving the use and integration of methods and evaluation designs.

STRATEGIC ACTIVITIES
In order to achieve these objectives, the TWG plans the following activities:

  • Annual workshops of the TWG in order to enable evaluators to exchange experience, best practices and lessons learned knowledge.
  • Develop a virtual platform for evaluators for knowledge exchange.
  • Propose and agree with the EES Board ona methodological theme/strand for the biennial EES conference.

Specific Contact Person in the EES Board

Estelle Raimondo, estelle.raimondo@gmail.com

List of TWG members (EES members in good standing)
Barnett Chris ITAD Ltd United Kingdom Email contact
Baysa Asham International Rescue Committee (IRC) Ethiopia Email contact
Bernabè Sabine European Investment Bank Luxembourg Email contact
Bockel Natalie France Email contact
Canu Eloisa Regione Autonoma della Sardegna Italy Email contact
Carbon Michael Carbon Italy Email contact
Charlot Clement France Email contact
Copestake James University of Bath United Kingdom Email contact
Davies Rick United Kingdom Email contact
Delahais Thomas France Email contact
Eminovic Jasenko World Vision International Bosnia and Herzegovina Email contact
Faust Joerg Germany Email contact
Guertin Michelle CIMMYT Mexico Email contact
Gupta Taruna Cyprus Email contact
Hall Pascale United Kingdom Email contact
Hofman Joanna RAND Europe United Kingdom Email contact
Hurteau Marthe Universite du Quebec Canada Email contact
Inyang Janet Family Health International Nigeria Email contact
Karijo Eddah Kenya Email contact
Kuprii Volodymyr Ukraine Email contact
Lacerda Luis Eduardo Portugal Email contact
Lappo Alena Belarus Email contact
Mevissen Jos Netherlands Email contact
Mutambara Diliah Zimbabwe Email contact
Nanyanzi Marie Uganda Email contact
Obser Andreas ASEAN – GTZ Germany Email contact
Ojok Reagan Ronald Uganda Development Bank Kenya Email contact
OSEI-MENSAH Emmanuel Ghana Email contact
Potluka Oto Switzerland Email contact
Raimondo Estelle Raimondo Estelle USA Email contact
REBOLLEDO DELLEPIANE Miguel Angel USA Email contact
Remr Jiri Czech Republic Email contact
Rishko-Porcescu Antonina EvalYouth, EvalYouth ECA (Eastern Europe, Central Asia, South Caucasus) Moldova (Republic of) Email contact
Sánchez Victoria Red2Red Consultores, S.L. Spain Email contact
Sever Rita Israel Email contact
Tenna Fabrizio Centrale Valutativa S.r.l Italy Email contact
Ton Giel Institute of Development Studies United Kingdom Email contact
Uusikylä Petri Frisky & Anjoy Finland Email contact
Weiss Lauren Independent Consultant France Email contact
Winter Kate Kate Winter Evaluation LLC USA Email contact
Current Work Plan
To be added soon…
Resources to download
Additional resources for TWG members only

To be added soon…