Authors: Dr. Anne Stephens and Dr. Ellen Lewis

EES in 2018 was a big event for us and our team of colleagues. We launched the Inclusive Systemic Evaluation for Gender Equality, Environments and Marginalized voices (ISE4GEMs), a systemic evaluation guidance with the tag: ‘a new approach for the SDG Era’. With a sense that this somehow sat on the edge of a precipice, Professor Thomas Schwandt provided a keynote address that Zenda Ofir likened to a “keynote that is Innovative . . . Thought-provoking . . . Visionary” on the emergence of post-normal evaluation, an adaptation of Funtowicz and Ravetz’s post-normal science where scientific reasoning can no longer be relied upon to generate indisputable conclusions. Evoking our excitement is his reliance on systems thinking and complexity science to build a case for the emergence of a post-normal evaluation practice – the type of evaluation approach needed if we accept that we do, in fact, live in post-normal times.

While compelling, scholarly, and later published in the journal Evaluation, Schwandt did not finish on a hopeful note that evaluation would shift to meet the new times and ,by implication, embrace evaluation approaches such as our ISE4GEMs. In his published version of the speech, he finishes the article with: “I am not wildly optimistic that these indications of a post-normal evaluation will be fully realized soon…”

But this was before the 2020 pandemic.

What’s changed is, well, everything, and not so much, depending on who you’re listening to or working for. It’s either a ‘new normal’ or ‘return to normal’. One seeks to open up our thinking to great opportunities for seismic shifts that put an end to the modernity ; others want desperately for a return to business-as-usual. These divergent views lead to very different understandings of democracy, culture and Nature.

We err on the side of viewing this pandemic and these any-but-normal times as a great opportunity for transformative and systemic change. And to do this, we encourage systemic thinking as an approach to collaboratively structure problems, manage and resolve them.  There are literally hundreds of methods, tools and academic texts. We did the work of sifting through many of these when we formulated the ISE4GEMs Guide.

So, what exactly is ‘systemic evaluation’?  In the ISE4GEMs guide, we make an important differentiation between ‘systematic’ and ‘systemic’. Put very simply, to think systematically, one engages in a process that can be thought of as, predictable, often times linear, thorough and controlled, as a way to think about parts of a system that may interact. By contrast, to think systemically encourages us to be more critical about how those parts interact and the multiple perspectives that could be influential in shaping interaction. What are the opportunities, constraints and relationships of parts within a system that tell us much about the system as a whole?  We write:

‘Systemic evaluation asks questions to capture the conditions and changes relevant to an intervention, the changes it produces and opportunities for learning and empowerment. In the course of an evaluation, one may uncover what else was going on—the external effects, spill over of other efforts or policies, uncontrolled events (such as political conflict or environmental disasters), or unexpected facilitators or inhibitors of change that may or may not have been part of the original plan.’ (ISE4GEMs, p. 8)

The ISE4GEMs is more than a description of systems thinking. It provides a primer in systemic evaluation for practice. Our intention is to be your thought partner, to help you reflect on the work you are being asked to do as you unfold your understanding of the evaluation. We describe a novel intersectional framework (the GEMs Framework) to foster authentic inclusive thinking. We then provide guidance with the detailed steps one might take to do a systemic evaluation. We created tools to work with and resources to draw on. Further to this, we’ve used the approach, as we were writing it. So many examples from our practice and others are embedded in the guide.

Why is this different?  Because there are not too many guides that have reviewed the literature and carefully chosen techniques, strategies, ideas and concepts from across the disciplines of evaluation, systems sciences and feminism, spilling over into a range of social and environmental science disciplines as well. We have selected key ideas and established a framework that generates practitioners’ knowledge and capacity to ‘do’ systems thinking, rather than just have a better general awareness of the theory.

Get involved

We invite all evaluators and practitioners, emerging or experienced, who are interested in using the ISE4GEMs to use the approach, and let us know, as your thought partners, what works, what doesn’t and how we can make the approach better.

We have an active research program and global network of practitioners to investigate inclusive systemic evaluation. We want people to experience the approach and tell us how it needs to be improved, which includes the ISE4GEMs toolkit, theory (Part A) and guide (Part B). Our community of practice will shape the ISE4GEMsV2 and together we can make better sense of our complex world.

Contact us at www.ethosofengagement.com and contact@ethosofengagement.com

This blog builds on an earlier version published on the BetterEvaluation website in 2018.