A couple of weeks ago I participated in the 2026 Annual Conference of the European Forum for Studies on Research and Innovation, which is primarily a community of science, technology, innovation (STI) scholars and practitioners, and increasingly evaluators as well. Sitting in session after session over those three days at the Universitat Politècnica de València, I kept recognizing questions I had been grappling with for years, now surfacing in a different disciplinary register. That sense of recognition is what I want to try to trace here.
The conference brought together scientists, STI researchers, social scientists, evaluators, and practitioners from firms, think tanks, universities, and public institutions to examine how science and innovation contribute to society. Seven sessions were dedicated to transformative evaluation in relation to system change, policy learning, social innovation, inclusion, and sustainable transitions. I found that significant space attributed to evaluation striking. It told me something about how the field is moving: evaluation is no longer being treated as a downstream exercise, something you bolt on after the policy has run its course. As Molas-Gallart and colleagues have pointed out, it is increasingly understood as part of how transformative policy is framed, tested, questioned, and revised. That shift matters—and I think it matters especially for us, as a community, to pay attention to it.
What is at stake is Transformative Innovation Policy, or TIP: a set of ambitions that push well beyond the usual metrics of R&D investment, patents, or firm competitiveness, toward questions of sustainable transitions, territorial resilience, and social inclusion. But I left Valencia with a nagging unease. These ambitions are under pressure. The European Commission’s Competitiveness Compass signals a reorientation toward strategic autonomy, industrial performance, and the scaling of technologies and firms. These are legitimate priorities. But they can narrow the space for the harder questions: who benefits from innovation, who bears its risks, and what kinds of institutional change are required for transitions to happen. My sense, sitting with those conversations in Valencia, is that transformative evaluation is one of the few tools capable of keeping those questions alive. Let me try to explain why.
Can a theory of change actually change how we think?
Theories of change came up constantly—in plenary discussions, in paper presentations, and in informal conversations. I have worked with them long enough to know they can be genuinely useful and genuinely hollow in equal measure.
The problem, as several speakers put it, is not that theories of change are wrong. It is that they are often too tidy, and abstract. They map activities to outputs to outcomes to impacts as if the pathway were already settled – as if we knew in advance why it would work, for whom, and under what conditions. They can also imply a single route to change when the actors involved are coming from very different organizational, technological, and territorial positions. That version of a theory of change becomes a compliance document, not a thinking tool.
What I kept returning to—and what Funnell and Rogers’ Purposeful Program Theory has long argued—is that a theory of change should be an instrument of inquiry, not a diagram. It should force us to make assumptions visible so we can interrogate them. It should help us understand why some pathways activate and others do not. And along these lines, the conference added something I found genuinely clarifying: a theory of change becomes most powerful when it is connected not just to single-loop learning—are we doing things right?—and double-loop learning—are we doing the right things?—but to triple-loop learning: how do we decide what is right in the first place?
That third question requires asking who defines the transformation, whose knowledge shapes the policy theory, and whether the direction of change has genuine social and democratic legitimacy. Those are uncomfortable questions to bring into an evaluation. But I am increasingly convinced they are the ones that matter most —and yet, easily dropped when evaluations are under time and resource pressure.
Systems thinking is seductive—and I worry about that
The systems thinking sessions were where I felt most intellectually alive—and most cautious. Systems approaches allow evaluators to trace interdependencies, feedback loops, and effects that play out across organizations, territories, and socio-technical systems. That is exactly the kind of evaluative reach that transformative policy demands.
But I kept asking myself: at what cost? Complex system models are expensive to build and harder to maintain. For instance, the literature include sophisticated system maps for sustainability transitions that are not systematically assessed and updated. And a map that cannot be updated is not an evaluative architecture but a snapshot.
What I came away thinking is that the design question matters as much as the modelling question. The real challenge is not how to map the whole system. It is how to build monitoring, evaluation, and learning architectures that can be sustained—anchored in specific R&I investments, selective enough to be usable, and designed to revise themselves as the system changes. That probably means following a limited number of critical mechanisms and signals of change, combining administrative data, surveys, network analysis, qualitative inquiry, and periodic case studies—not to capture everything, but to stay close to the processes that matter most for transformation. I left that strand of the conference with more questions than answers, which I think is probably the right outcome.
Cases taught me things I could not have found in a database
The case-based sessions focusing on specific interventions or programs felt, in some ways, like coming home. This is the kind of evaluative work I trust most—and not just for methodological reasons.
Rich cases show you things that indicators cannot. They show how actors interpret an intervention—which is rarely what the policy designers intended. They show where resistance emerges, how relationships shift, how informal coordination happens or breaks down. They make visible the trust, negotiation, and power dynamics that shape whether a policy lands. Andy Stirling’s critique in one of the plenary sessions regarding misleading technocratic precision in research evaluation resonated here: the appearance of rigor that comes from standardized indicators and other less participatory approaches can obscure the things that most need to be understood.
What I find particularly compelling for transformative evaluation is what case studies can do in place-based innovation policy. These are contexts where ultimate impacts are rarely achieved within an evaluable timeframe—and where reconstructing how actors coordinated, reasoned, and adapted is far more revealing than trying to attribute outcomes. Cases let you follow the theory of action as it unfolds in the endeavor of different stakeholders, preserving the contextual conditions that made them possible. That is not generalization in the statistical sense—it is explanation that can travel, carefully, across cases while remaining anchored to institutional and territorial difference. I find that kind of knowledge useful for transformative evaluation and usable for policy design.
The deep question of democracy
Transformative policies are not value neutral. They involve choices about whose needs are prioritized, which risks are deemed acceptable, and how costs and benefits are distributed. Evaluators who treat those choices as given—who accept the framing and measure performance within it—are not being neutral. They are lending their credibility to a particular direction of change.
This becomes especially uncomfortable when technocratic policymaking coexists with illiberal tendencies in political systems. I have been thinking about this for a while, reflecting on how expert knowledge in authoritarian contexts can be used to close debate rather than open it. I have argued elsewhere that evaluation under democratic erosion demands more than technical quality. It requires preserving spaces for plural evidence, for disagreement, for scrutiny. A democratic evaluative posture means sustained engagement with stakeholders—including those whose voices are excluded—and genuine transparency about the assumptions built into our evaluative frameworks.
Here, again, triple-loop learning means asking who gets to participate in the decision of what is right, and whether alternative forms of knowledge and experience can alter the direction of policy, or whether they are gathered and then set aside. I do not have a clean answer to that. But I think it is the question the evaluation community needs to sit with more honestly.
What I brought back from Valencia—and what I am hoping we can build
I left the conference with a clearer sense of why the conversation between EU-SPRI and EES matters—and what each community might genuinely offer the other.
EU-SPRI lives close to the conceptual and empirical debates that are reshaping what innovation policy is supposed to do. That proximity is valuable for EES, which risks engaging with transformative evaluation at a distance from the science, technology, and innovation systems it is meant to evaluate. At the same time, EES brings something EU-SPRI needs: a broad, plural evaluation community with deep experience in development, environment, education, health, social policy, organizational change, democracy, and governance. These are communities that have long worked with complexity, participation, qualitative inquiry, and the political dimensions of evidence—often in more difficult contexts than innovation policy typically faces.
Joint sessions, thematic working groups, comparative case research, and cross-community publications could help keep transformative evaluation from becoming a specialised conversation that loops back on itself. EU-SPRI can broaden the substantive horizons of evaluation by bringing science, technology, and innovation systems more fully into view. EES can strengthen the methodological, democratic, and cross-policy foundations of that work. Together, the two communities might be better equipped to ask not simply whether innovation contributes to society—but how, for whom, under what conditions, and toward which forms of transformation. Those are the questions I came back from Valencia still turning over. I hope we can turn them over together.
Bio
Mita Marra is an Associate Professor of Political Economy and Innovation Policy Evaluation at the University of Naples, Italy. She is a former EES Board Member and a Past President of the Italian Evaluation Association (AIV).