Riitta Oksanen is the President of EES. She works as a senior advisor at the Finnish Ministry for Foreign Affairs, Development Evaluation Unit. In this post Riitta triangulates with Fredrik Korfker former Chief Evaluator of the EBRD and now active as development finance consultant focusing on the private sector and Marvin Taylor-Dormond, current Director of Independent Evaluation for Financial, Private Sector and Sustainable Development at the World Bank Group.

Key points:

1. Market-oriented development initiatives, often supported by public funds, are an essential part of efforts to achieve the global development commitments. Evaluation in this field is ongoing – but is the service industry involved in the evaluation of social impact investments willing to work together with the development evaluation community?
2. Evaluating market-oriented development implies finding a balance between the rapid efficiency of the industry, and thinking through the systematic application of evaluation standards in this context. It is essential that evaluation approaches be aligned and responsive to the private sector/market-base nature of operations, instead of lazily adapting public sector practices. 
3. Bringing together the stakeholders for dialogue is the best way forward, and urgently needed.

Riitta: Market –oriented development initiatives have an increasingly important role in achieving sustainable development. Evaluation, however, has traditionally been linked to public sector operations. Why and when is it important to also evaluate market-oriented initiatives?

Fredrik: Market-oriented development initiatives have indeed an increasingly important role in achieving sustainable development. During the Wilton Park Conference in July 2015 “New Frontiers for evaluation in an era of market oriented development” the growing social impact investment industry was discussed and conclusions were reached about the importance of rigorous evaluation practices that need to enhance transparency and credibility of this industry. It is also important to refer in this respect to the G20 meeting in London May 2014 where the importance of social investment was highlighted as a driver for development. The OECD was asked to prepare a report to get a better view of the industry: Social Impact Investment-building the evidence base” (http://www.oecd.org/sti/ind/social-impact-investment.pdf).  It gives an indication of the changing views of donor countries having been involved in development financing over many years and stimulating market-oriented approaches in development financing. During the Wilton Park conference participants, many being evaluators with a development background, stressed the importance of thorough and independent evaluation of social impact investment activities. The rigorous evaluation methodologies and practices developed over many years for the public sector could also be used for evaluating social impact investments and it was felt that a dialogue should be stimulated between different groups of evaluators to learn from each other. In respect of your last question Riitta, “why and when is is it important to also evaluate market-oriented initiatives”, I want to respond that (a) whenever public funds are involved in financing market-oriented operations, as is the case in private sector financing by multilateral development banks (MDBs), or (b) when for instance tax incentives are provided by governments to stimulate the public to invest in social impact investment operations, it is essential to secure transparency and to demand rigorous evaluation to take place, to guarantee the necessary transparency to the social impact investors and the general public.

Marvin:   In capitalist societies – which is the case of the majority of world societies today- market oriented initiatives are actually the key ingredient of development.  For instance, 9 out of 10 jobs worldwide are created by market-oriented activities. The recognition of the role of the private sector in development had been absent until very recent. Today, the situation has changed significantly. The most recent global commitments on SDGs and Climate Change, for instance call for an active promotion of and partnership with the private sector, to achieve such goals. In response to this changing context, there has been a renewed interest by multilateral development banks, bilateral agencies and impact investors in promoting development through market-based interventions. Now how to evaluate these activities? It is correct as you indicate that evaluation has traditionally been associated with public sector initiatives. However, within the context of the Evaluation Cooperation Group, Multilateral Banks that were created to work through the private sector, such as IFC and EBRD have engaged in the creation of a set of standards to evaluate private sector or market based projects.  This work has been precisely led by Fredrik, Bill Stevenson my predecessor in IFC and by me.  The standards recognize the specificity of these types of interventions, such as the competitive environment in which the private sector operates, the payment that beneficiaries have to make to have access to services or goods, the importance of the financial sustainability of interventions, and the criticality of third party effects of these interventions, i.e. their social and environmental effects. In addition to this, as Fredrik indicates, parallel to the growing trend of the impact investment industry, an entire new practice of assessment and guidelines for evaluating the results of these investments is emerging, often led by the accounting community as was the case at the initial stages of evaluation in the public sector many years ago and by private sector associations.  My concern regarding these initiatives has to do with the rigor and alignment or lack thereof of these practices and guidelines as well as with the limited role of the evaluation community in this development. I believe that the evaluation community should urgently start paying attention to the growing use of market oriented interventions to promote development and organize itself to play a strong role in the development of tools to assess their results.

Riitta: What are the biggest challenges in evaluating development processes with private sector engagement? Do not need new tools and concepts, do we need new ways of thinking?

Marvin:  In my experience, the biggest challenge is twofold: to get evaluators to switch their traditional mindset and connect with the nature of private sector interventions and to develop the appropriate frameworks to assess these types of interventions. On the first, simplicity and speed of the evaluation process, as demanded by the competitive framework of private sector operations is a must. On the second, in evaluation, our frameworks have to follow the nature of and circumstance of the object of evaluation.  Therefore, a good framework should recognize the characteristics of private sector interventions instead of lazily trying to adapt frameworks created to assess public sector project for evaluating private sector operations.  I think that there is a long way to go in this respect. I live through these difficulties every day, within our own organization as well as in dealing with evaluators from other agencies trained to see development and evaluation exclusively through the lens of the public sector.  But as I said, the growing industry will not wait for anyone who may not be willing or ready to understand the new reality. It is the private sector and it will create its own instruments with or without our active participation.

Fredrik: One of the important areas where evaluating of development processes with private sector takes place is in the multilateral development banks (MDBs). From the start of these institutions (World Bank Group and four regional development banks) ex post evaluation of public as well as private sector operations has been essential assessing performance in respect of their mandates and in holding the institutions accountable to their owner-governments and the general public. In respect of private sector evaluation, I have always put emphasis on project evaluation, i.e. evaluating individual projects through carrying our field visits and learning on the spot how the project is doing and what impediments are there to fulfilment of objectives.  Also bottlenecks can be spotted during such evaluations and sometime violations in respect of business ethics and corruption, which would remain hidden if no field visits would take place.  Also in respect of gathering quality lessons learned on projects, these field visits are crucial. The tendency in MDB evaluation departments, when evaluating private sector operations is to do so-called “higher level” evaluation and to abolish the expensive field visits of projects and concentrate only on validation of self-evaluation reports by operational staff. The private sector is not necessary keen to be the object of evaluation, but during the interaction with clients it is important to make clear that MDB financing is focusing on enhancing development impact and transition (the latter, as is the case for the EBRD) and that each of the MDBs involved in private sector financing should be additional, i.e. should have strong conditionality (environmental/transparency), should be in principle more expensive than commercial banks and should focus on value addition of the institution.

Riitta: How can private sector demand for evaluation be best encouraged?  How do we get to “My accounts are audited AND My results are evaluated”?

Fredrik: It is important that in particular for a social impact investment industry, regulations should be as such that rigorous evaluation is required and that the industry has proper control mechanisms. Cross-fertilization from the development evaluators towards the consultant firms and their representatives responsible for evaluating social investment operations, through regular dialogues, in my view is essential. The industry responsible for social impact investment who commission the evaluations and the controlling authorities should also get involved in this debate.

Marvin:  I do agree with Fredrik. In the impact investment industry, there is no need to stimulate demand because the entire concept is based on the intention of having an impact. In other words, the double bottom line you suggest (profitability/sustainability and results) is inherent to the operations.  Here the trouble is with the development of good and rigorous evaluation frameworks and with the involvement of the evaluation community to meet the demand. On the other hand, among multilateral and bilateral agencies, the demand is also natural, for development and therefore working to meet a double or triple bottom line (including environmental sustainability) is their mission. The difficulty lies in developing the right instruments and the intellectual curiosity of evaluators to deal with private sector or market oriented activities as opposed to just public sector projects.

Riitta: Fredrik, you suggested above that it is important to get a dialogue going between development evaluators and representatives of the social investment industry, the firms involved in evaluation of the sector and also to talk to the regulators. How can we make this happen, what are the most important next steps? Can we use the EES Conference in Maastricht as a forum?

Fredrik: We should invite speakers from the social investment industry, i.e. some key players involved with market development of social impact investment, representatives from the big accounting firms involved in evaluation and some responsible regulators of that industry.  An important contact would be the GIIN, the Global Impact Investing Network that might have some suggestions in preparing the conference in respect of the social impact investment theme. We should indeed prepare a forum at the Conference on social impact investment.

Marvin: I agree with Fredrik:  we urgently need to stimulate a dialogue among investors, fund operators and the accounting community currently filling the gap of evaluation services. Among bilaterals and multilaterals, we need to continue the conversation on the importance of understanding that in our evaluation work, framework and method should follow object and therefore, that applying the same public sector framework to market oriented interventions is incorrect or inadequate.    The upcoming EES Conference can be actively and strategically used to motivate the dialogue. In addition, it can be used to examine the tools that are already in existence in ECG to deal with private sector/ market oriented interventions, and motivate their use and benchmarking among bilaterals and multilateral as well as among consultants. In Dublin the EES Private Sector Working Group under the leadership of Fredrik was very active and organized itself to produce a series of presentations on the subject. IEG of the World Bank Group was strongly supportive of this initiative. We should strive for an even stronger and focused effect in Maastricht.

Riitta: Many thanks for your visit to my blog! Let’s invite views from readers as an immediate response, and continue in Maastricht!