TWG 6 – Methods and Designs

Search on EES

TWG Leader

TWG Leader

Sven Harten

Presentation of TWG

Evaluations often deal with multiple and multi-dimensional evaluation questions and assess complex evaluation objects in changing contexts. This requires evaluators to apply different methods and to analyze different types of data. Additionally, the evaluation community is multi-disciplinary relying on a wide set of analytical procedures from the social sciences, natural sciences, applied sciences and humanities. The toolbox of available designs and methods is thus rich and diverse.

This EES Thematic Working Group deals with issues around evaluation methods and designs. Specifically, since the evaluation community agrees that there is a potential value added from integrating different methods in evaluations, the primary focus for the first four years (2019-2023) will be on how to apply and integrate multiple methods in mixed/multi-method designs. This concept note is therefore written with a focus on method integration in mind. The members of the TWG will decide in 2019 on a bi-annual work program (until 2021) and at the end of the second bi-annual (2021-2023) will decide whether to maintain or change the thematic focus of the TWG.

The relevance of setting up this working group builds upon the following observations:
1. No single method is able to cope with interventions that affect multiple levels of analysis, involve short- and longterm outcomes, and address complex systems with interacting components. The TWG supports the evaluation community in selecting and integrating suitable quantitative and qualitative methods.
2. Evaluation can benefit from rigorous causal inference. The TWG supports the identification of causal effects and causal mechanisms through integrated cross- and within-case designs.
3. There is potential in the increasing availability of mixed- and multi-method approaches to evaluation. The TWG contributes to clear evaluation designs by providing formal protocols.
4. Evaluations increasingly draw on new types of data (including big data such as geo-spatial imagery, transactional data, mobile phone records or internet/app data) and use of innovative analytical tools like machine-learning. The TWG promotes the integration of emerging data types and analytical techniques in complex evaluations.
The TWG sees an increased need for methodological exchange and learning on method integration.

With this Thematic Working Group we strive to improve the quality of evaluations and strengthen evidence-based decision making through better methods and designs. Specifically, our aim for the coming four years is advancing the systematic integration of different methods in evaluations in order to generate more valid and reliable evidence.


Objectives of TWG

The TWG “Methods and Designs” will be a collaborative space to share experiences, good practices and lessons learned round evaluation methods and designs while at least initially a particular focus will lie on systematically integrating different methods in evaluations. It builds on ongoing debates in mixed- and multi-method research within the evaluation discipline and in the broader social sciences. The TWG aims at bringing together evaluators and academics to exchange ideas and experience on innovative ways of combining methods and to develop systematic approaches of method integration in evaluation.

The main objectives of the TWG are to:

  • Facilitate methodological debate and exchange among EES members, specifically on the matter of method integration.
  • Bundle expertise and best practices in mixed-methods and multi-method approaches for evaluation.
  • Reconcile different methodological cultures in the evaluation community.
  • Transfer and adapt recent methodological innovations from the social sciences for the field of evaluation.
  • Work and advance towards a more systematic integration of multiple methods in evaluation.
  • Increase the availability of systematic approaches to method integration in evaluation.
  • Provide guidance for evaluators on the selection of appropriate methods and their integration.
  • Improve the integration of new types of data and data analysis in evaluation designs.

Accordingly the TWG helps bring together evaluators and academics interested in improving the use and integration of methods and evaluation designs.

In order to achieve these objectives, the TWG plans the following activities:

  • Annual workshops of the TWG in order to enable evaluators to exchange experience, best practices and lessons learned knowledge.
  • Develop a virtual platform for evaluators for knowledge exchange.
  • Propose and agree with the EES Board ona methodological theme/strand for the biennial EES conference.

Specific Contact Person in the EES Board

Estelle Raimondo,

List of TWG members (EES members in good standing)
Anta Akuro Germany 
ARHIN-SAM Kwaku Friedensau Adventist University Germany 
Barnett Chris ITAD Ltd United Kingdom 
Baysa Asham International Rescue Committee (IRC) Ethiopia 
Bernabè Sabine European Investment Bank Luxembourg 
Bockel Natalie France 
Canu Eloisa Regione Autonoma della Sardegna Italy 
Carbon Michael Carbon Italy 
Copestake James University of Bath United Kingdom 
Davies Rick United Kingdom 
Delahais Thomas France 
Eminovic Jasenko World Vision International Bosnia and Herzegovina 
Faust Joerg Germany 
Guertin Michelle CIMMYT Mexico 
Hofman Joanna RAND Europe United Kingdom 
Hurteau Marthe Universite du Quebec Canada 
Inyang Janet Family Health International Nigeria 
Karijo Eddah Kenya 
Lacerda Luis Eduardo Portugal 
Lappo Alena Belarus 
Mevissen Jos Netherlands
Mutambara Diliah Zimbabwe 
Nanyanzi Marie Uganda 
Obser Andreas ASEAN – GTZ Germany 
Ojok Reagan Ronald Uganda Development Bank Kenya 
OSEI-MENSAH Emmanuel Ghana 
Potluka Oto Switzerland 
Raimondo Estelle Raimondo Estelle USA 
Remr Jiri Czech Republic 
Rishko-Porcescu Antonina EvalYouth, EvalYouth ECA (Eastern Europe, Central Asia, South Caucasus) Moldova (Republic of) 
Sánchez Victoria Red2Red Consultores, S.L. Spain 
Sever Rita Israel 
Tenna Fabrizio Centrale Valutativa S.r.l Italy 
Ton Giel Institute of Development Studies United Kingdom 
Uusikylä Petri Frisky & Anjoy Finland 
Weiss Lauren Independent Consultant France 
Winter Kate Kate Winter Evaluation LLC USA 
Current Work Plan
To be added soon…
Resources to download
Additional resources for TWG members only

To be added soon…