This is just one of the recurring themes Dr. Ralph Renger addresses in his new release, System Evaluation Theory: A blueprint for practitioners evaluating complex interventions operating and functioning as systems.
“The focus in evaluation on wicked problems is misplaced and harmful” says Renger. “We don’t evaluate wicked problems; we evaluate the complex interventions designed to address them.”
Drawing on his psychology training Renger goes on to argue that if evaluators believe something is “wicked” then one of two things typically happens: either evaluators revert back to what they know, like using logic models to evaluate complex interventions, or they just abandon ship altogether.
The book is filled with practical examples from Renger’s own evaluation practice and everyday examples to which anyone can relate. For example, one chapter uses the example of family interactions to illustrate system properties and principles central to completing an evaluation of a complex intervention. Renger also draws on his emergency preparedness training using examples from the pandemic; something to which the entire world can arguably relate.
The book is logically organized and easy to follow. System Evaluation Theory (SET) begins by establishing whether the intervention is indeed operating and functioning as a system and not as Meadows (2008) so succinctly described it “just a bunch of stuff”.
Two chapters are then devoted to illustrating how to define the complex intervention. These chapters emphasize how an understanding of interdependence drives the methods for defining the complex intervention.
This is followed by chapters explaining how to evaluate the efficiency of the complex intervention by applying system principles, like feedback loops, reflex arcs, and cascading events.
One of the more interesting chapters is on evaluating the effectiveness of complex interventions, wherein Renger describes why the system property of emergence is the most appropriate measure of effectiveness for a complex intervention, and explains why equating emergence to long term outcomes such as those found in logic models is misplaced.
This book is intended for evaluation practitioners and those wanting to learn how to conduct an evaluation of a complex intervention using a systems approach. Renger, an academic for 27 years, takes issue with many academics’ portrayal of complexity, noting that the discussion often happens without an understanding of systems. Thus, the discussions are often as Checkland (2000) described, purely academic in every pejorative sense.
“Evaluating complex interventions is not complicated (i.e., difficult)” says Renger, “it will take time and patience, but it can definitely be done by applying the 3 SET steps”.
The book is currently available for sale online through www.infoagepub.com/products/System-Evaluation-Theory and through Amazon.