TWG 6 – Methods and Designs

Search on EES

TWG Leader

TWG Leader

Sven Harten

Presentation of TWG

Evaluations often deal with multiple and multi-dimensional evaluation questions and assess complex evaluation objects in changing contexts. This requires evaluators to apply different methods and to analyze different types of data. Additionally, the evaluation community is multi-disciplinary relying on a wide set of analytical procedures from the social sciences, natural sciences, applied sciences and humanities. The toolbox of available designs and methods is thus rich and diverse.

This EES Thematic Working Group deals with issues around evaluation methods and designs. Specifically, since the evaluation community agrees that there is a potential value added from integrating different methods in evaluations, the primary focus for the first four years (2019-2023) will be on how to apply and integrate multiple methods in mixed/multi-method designs. This concept note is therefore written with a focus on method integration in mind. The members of the TWG will decide in 2019 on a bi-annual work program (until 2021) and at the end of the second bi-annual (2021-2023) will decide whether to maintain or change the thematic focus of the TWG.

RELEVANCE
The relevance of setting up this working group builds upon the following observations:
1. No single method is able to cope with interventions that affect multiple levels of analysis, involve short- and longterm outcomes, and address complex systems with interacting components. The TWG supports the evaluation community in selecting and integrating suitable quantitative and qualitative methods.
2. Evaluation can benefit from rigorous causal inference. The TWG supports the identification of causal effects and causal mechanisms through integrated cross- and within-case designs.
3. There is potential in the increasing availability of mixed- and multi-method approaches to evaluation. The TWG contributes to clear evaluation designs by providing formal protocols.
4. Evaluations increasingly draw on new types of data (including big data such as geo-spatial imagery, transactional data, mobile phone records or internet/app data) and use of innovative analytical tools like machine-learning. The TWG promotes the integration of emerging data types and analytical techniques in complex evaluations.
The TWG sees an increased need for methodological exchange and learning on method integration.

VISION
With this Thematic Working Group we strive to improve the quality of evaluations and strengthen evidence-based decision making through better methods and designs. Specifically, our aim for the coming four years is advancing the systematic integration of different methods in evaluations in order to generate more valid and reliable evidence.

 

Objectives of TWG

MISSION STATEMENT
The TWG “Methods and Designs” will be a collaborative space to share experiences, good practices and lessons learned round evaluation methods and designs while at least initially a particular focus will lie on systematically integrating different methods in evaluations. It builds on ongoing debates in mixed- and multi-method research within the evaluation discipline and in the broader social sciences. The TWG aims at bringing together evaluators and academics to exchange ideas and experience on innovative ways of combining methods and to develop systematic approaches of method integration in evaluation.

The main objectives of the TWG are to:

  • Facilitate methodological debate and exchange among EES members, specifically on the matter of method integration.
  • Bundle expertise and best practices in mixed-methods and multi-method approaches for evaluation.
  • Reconcile different methodological cultures in the evaluation community.
  • Transfer and adapt recent methodological innovations from the social sciences for the field of evaluation.
  • Work and advance towards a more systematic integration of multiple methods in evaluation.
  • Increase the availability of systematic approaches to method integration in evaluation.
  • Provide guidance for evaluators on the selection of appropriate methods and their integration.
  • Improve the integration of new types of data and data analysis in evaluation designs.

Accordingly the TWG helps bring together evaluators and academics interested in improving the use and integration of methods and evaluation designs.

STRATEGIC ACTIVITIES
In order to achieve these objectives, the TWG plans the following activities:

  • Annual workshops of the TWG in order to enable evaluators to exchange experience, best practices and lessons learned knowledge.
  • Develop a virtual platform for evaluators for knowledge exchange.
  • Propose and agree with the EES Board ona methodological theme/strand for the biennial EES conference.

Specific Contact Person in the EES Board

Estelle Raimondo, estelle.raimondo@gmail.com

List of TWG members (EES members in good standing)
Anta Akuro Germany akuro.anta@fu-berlin.de 
ARHIN-SAM Kwaku Friedensau Adventist University Germany kwaku.arhin-sam@thh-friedensau.de 
Barnett Chris ITAD Ltd United Kingdom chris.barnett@itad.com 
Baysa Asham International Rescue Committee (IRC) Ethiopia asham.assazenew@rescue.org 
Bernabè Sabine European Investment Bank Luxembourg z.arencibia@eib.org 
Bockel Natalie France natalie.bockel@gmail.com 
Canu Eloisa Regione Autonoma della Sardegna Italy eloisa.canu@gmail.com 
Carbon Michael Carbon Italy michael.carbon@outlook.com 
Copestake James University of Bath United Kingdom j.g.copestake@bath.ac.uk 
Davies Rick United Kingdom rick.davies@gmail.com 
Delahais Thomas France tdelahais@quadrant-conseil.fr 
Eminovic Jasenko World Vision International Bosnia and Herzegovina Jasenko_Eminovic@wvi.org 
Faust Joerg Germany joerg.faust@deval.org 
Guertin Michelle CIMMYT Mexico m.guertin@cgiar.org 
Hofman Joanna RAND Europe United Kingdom joanna.hofman@gmail.com 
Hurteau Marthe Universite du Quebec Canada hurteau.marthe@uqam.ca 
Inyang Janet Family Health International Nigeria inyang.janeti@gmail.com 
Karijo Eddah Kenya eddahkan@gmail.com 
Lacerda Luis Eduardo Portugal luis.lacerda2@gmail.com 
Lappo Alena Belarus alena.lappo@gmail.com 
Mevissen Jos Netherlands evaluatie-advies@xs4all.nl
Mutambara Diliah Zimbabwe diliah.mutambara@gmail.com 
Nanyanzi Marie Uganda mnanyanzi@gmail.com 
Obser Andreas ASEAN – GTZ Germany andreas.obser@me.com 
Ojok Reagan Ronald Uganda Development Bank Kenya reaganojok@yahoo.co.uk 
OSEI-MENSAH Emmanuel Ghana emmanueloseimensah@ymail.com 
Potluka Oto Switzerland oto.potluka@gmail.com 
Raimondo Estelle Raimondo Estelle USA estelle.raimondo@gmail.com 
REBOLLEDO DELLEPIANE Miguel Angel USA miguelarebolledo@yahoo.com 
Remr Jiri Czech Republic remr@centrum.cz 
Rishko-Porcescu Antonina EvalYouth, EvalYouth ECA (Eastern Europe, Central Asia, South Caucasus) Moldova (Republic of) antonina.rishko@gmail.com 
Sánchez Victoria Red2Red Consultores, S.L. Spain vse@red2red.net 
Sever Rita Israel ritasever9@gmail.com 
Tenna Fabrizio Centrale Valutativa S.r.l Italy fabrizio.tenna@gmail.com 
Ton Giel Institute of Development Studies United Kingdom g.ton@ids.ac.uk 
Uusikylä Petri Frisky & Anjoy Finland petri.uusikyla@frisky.fi 
Weiss Lauren Independent Consultant France lweiss@capacity.ca 
Winter Kate Kate Winter Evaluation LLC USA kate@katewinterevaluation.com 
Current Work Plan
To be added soon…
Resources to download
Additional resources for TWG members only

To be added soon…