Riitta Oksanen is the Vice-President of the EES. She works as a senior advisor at the Finnish Ministry for Foreign Affairs, Development Evaluation Unit. In this post Riitta diablogues with Rakesh Mohan who is the Director of the Office of Performance Evaluations (OPE), an independent and nonpartisan agency of the Idaho State Legislature. He is a former board member of the American Evaluation Association (AEA) and currently serves on its Evaluation Policy Task Force. Riitta and Rakesh started discussing the topic of building partnerships in evaluation when they first met at the 2014 AEA conference in Denver.

Key Points

1. Evaluation is a service function. It only becomes meaningful when it meets the information demands and when the results are used.
2. Independence in evaluation becomes a curse unless complemented with strong partnerships that ensure relevance and responsiveness.
3. Because evaluations involve making judgment about prioritization, distribution, and use of resources, evaluations will always be inherently political.

Riitta: Rakesh, in Denver, you talked about building new partnerships to promote evaluation use among policymakers. In order to accomplish that, you said evaluators would need to broaden their discussions at conferences beyond evaluation methods and results. What did you mean by that?

Rakesh: At the closing plenary session of the 2013 AEA conference, I talked about the 4 Ps that are missing in discussions at AEA conferences. These 4 Ps are policymakers^[1], politics, the public, and the press.

Rarely do we see policymakers at AEA conferences. Of more than 3,000 attendees at the AEA conference in Denver this year, I believe there was only one policymaker – I had invited him to participate in a panel discussion. Furthermore, the politics of evaluation is seldom discussed at AEA conferences. There seems to be a myth among many evaluators that evaluation has nothing to do with politics. They believe they need to stay away from politics in order to maintain their credibility as independent and objective evaluators.

Now let’s talk about the public. We evaluators are passionate about making this world a better place, striving toward an equitable and sustainable future. However, people outside our conference hotels do not know what evaluation is or what evaluators do. Teachers, accountants, physicians, engineers, social workers do not need to explain to anyone what their professions are all about – but evaluators do. This is because we evaluators primarily talk among ourselves and do not interact much with others outside of our profession.

At this year’s closing plenary in Denver, Past AEA President Rodney Hopson asked whether we make a positive footprint in cities where we hold annual conferences. Does anyone in Denver know that we are here? Did Washington, DC know that we were there last year? Does anyone in Chicago know that AEA is coming to their community in 2015? In essence, he challenged us to think about the larger purpose of our profession and the association.

The fourth P is the press. More than 3,000 evaluators gathered in downtown Denver, but do we know if the local press covered our conference? Did they publish any story about us? Of the hundreds of conference sessions held, did we have something worth sharing with people outside the hotel? If yes, then we should have reached out to the press because the press could be a link between us and the public.

Unless we discuss these 4 Ps in greater depths at our conferences, in our journals, and in our classrooms, I do not believe we can have the kind of impact we would like to see through our evaluations in moving toward an equitable, sustainable world.  Evaluation is inherently linked to the 4 Ps. We must demonstrate the relevance of evaluation to policymakers, the public, and the press. Evaluators do not have the authority to make public policies; policymakers do. Therefore, we have no choice but to engage policymakers in evaluation activities. We should reach out to the press, which can carry our evaluation message to the public – policymakers tend to pay attention to their constituents.

Riitta, you too have emphasized the importance of working with political decision makers, and more broadly with colleagues who are responsible for evaluation demand and use, including commissioners of evaluation? Why do you think this is not automatically happening?

Riitta: One reason may be the curse of the principle of independence. When independence starts dominating the evaluation function, it very easily leads to self-centred isolation and irrelevance. Too often we forget that evaluation is guided not by a single but a set of principles. It is important to complement independence with relevance and responsiveness. Evaluation is a sport where you can only win a gold medal if you master this triathlon. J

I also think that we need to learn how to work with important partners beyond our own community. A part of this is to communicate in a non-jargon language. Rakesh, you have been successful in working with politicians. What lessons can you share on how best to approach non-evaluators?

Rakesh: Riitta, you are right about the curse of the principle of independence. Many evaluators are concerned that they will lose their independence, and hence their credibility, if they engage with policymakers. However, to conduct evaluations that are used by policymakers, we have to be relevant and responsive to their information needs.

Responsiveness involves understanding the political context of the evaluation; identifying key stakeholders and knowing competing, conflicting, and complementing interests of those stakeholders; and establishing relationships and building trust with key policy actors (includes policymakers and those who have access to policymakers, and those who influence the policymaking process). I do not believe we have a choice between independence and responsiveness; we have to do our best to maximize both and to know when and where to draw the line when independence is threatened so as not to compromise our professional standards and ethics.^[2]

Riitta: Agree. I think it is important for evaluators to remember that they are only evaluators. There are situations where evaluators, for example, working in the area of planning come very close to assuming the role of the planners. Evaluators should remember their role as visitors in processes that are owned and hosted by other professionals. Evaluation is a service function to these colleague professionals. Evaluators cannot and do not need to master everything. Partnerships are all about trust among professionals and reciprocal learning.

What would it take to be successful in this beyond-evaluation approach?

Rakesh: To be successful, I believe we will need to step outside our comfort zone and reach out to those who make and influence public policies that affect our lives and our environment. It requires building new partnerships with non-evaluation organizations that have access to policymakers and understand the complexities of public policy environments. These partnerships must be forged without sacrificing important principles of evaluation, such as independence, non-partisanship, and transparency in the public interest.

Riitta: How do you see the role of VOPEs or the Voluntary Organisations for Professional Evaluation (i.e., evaluation societies and associations) in building new partnerships?

Rakesh: Evaluation societies and associations across the globe are the perfect vehicle to promote evaluation use among policymakers. Individual evaluators, evaluation offices, and evaluation consulting firms do not have the needed capacity to reach out to policymakers and the public. However, VOPEs working together can. For example, Asian and African evaluation associations in collaboration with the EvalPartners are now making efforts to engage policymakers. The involvement of parliamentarians at the European Evaluation Society’s biennial conference in Dublin is an excellent example of what needs to be done worldwide on a consistent basis.

Thank you much, Riitta. I’m so glad we met in Denver. It was a pleasure to discuss with you this important topic of building new evaluation partnerships. That, in my view, is the evaluation future.