M&E in Portuguese Speaking Countries: moving the agenda forward

 

 

What do you know about Monitoring and Evaluation (M&E) in Lusophone countries, that is, countries where Portuguese is an official language?

 

Angola, Brazil, Cabo Verde, Equatorial Guinea[1], Guinea-Bissau, Mozambique, Portugal, São Tomé and Príncipe and Timor-Leste are countries, where M&E practices can be strengthened.

 

The Lusophone world

 

 

In 2018, a group of M&E Lusophone champions^[2] formed a task force to raise awareness on this issue. This group gathered at a roundtable to discuss “The road ahead for evaluation in the Lusophone world: resilience and action in critical times” during the European Evaluation Society (EES) biannual conference in Thessaloniki.

 

The roundtable gathered CLEAR Brazil and Lusophone Africa,  the President of the African Evaluation Association (AfrEA), Caroline Heidder (former Director of IEG/World Bank) and other reputable evaluation practitioners from Lusophone Africa. Since then, this conversation expanded and convened other stakeholders from Lusophone countries, as well as international organizations..

 

The Thessaloniki discussions galvanized participants around the urgency of this agenda, which demands a regional and a country comprehensive diagnosis as the first step to address the strengthening of M&E in Lusophone countries. Other steps include fostering public sector demand for evaluation for more evidence-based policy-making, enhancing academia’s support to evaluation capacity development, creating a local Portuguese speaking class of professional evaluators, and getting more donors involved in country-led sector initiatives.

 

During the roundtable, three challenges were identified across this range of countries: a fragmented evaluation field, lack of strong professional organizations dedicated to evaluation, lack of capacity to conduct advocacy and influence policy-making. These limitations are essentially caused by a lack of human and institutional capacity, insufficient financial and human resources, a reactive institutional culture, and lack of political commitment.

 

Common challenges in M&E in the Lusophone world

 

The main findings of this roundtable can be grouped into five areas: 1) the relevance of evaluation in each country´s national context; 2) evaluation demand; 3) evaluation supply; 4) evaluation capacity; 5) and knowledge management and sharing in evaluation. Each country presents different features which are briefly summarized below.

 

1. The relevance of Evaluation in each country´s national context

The cross-cutting role of evaluation needs to be better understood and supported in all Lusophone countries. In Brazil, the relevance of evaluation is widely recognized and promoted at the federal and provincial levels, with active evaluation networks creating awareness and promoting M&E activities (e.g. Rede Brasileira de Monitoramento e Avaliação). CLEAR Brazil and Lusophone Africa has supported different initiatives and promoted capacity building in articulation with public entities. In Cabo Verde, the need and worth of evaluation is also recognized in government, line Ministries and academia, with different initiatives taking place. In Mozambique, evaluation has received during the past decade growing recognition, with M&E present in the current Plano Quinquenal do Governo[3] through its Strategic Matrix which cascades down to all Ministries. In Timor-Leste, evaluation is mostly linked to the implementation of Timor-Leste´s Plano Estratégico de Desenvolvimento 2011-2030 and the Sustainable Development Goals (SDGs), otherwise mostly donor-related. In Portugal, it still continues to be marginal, done mostly through Instituto Camões and Agência para o Desenvolvimento e Coesão (the former responsible for the management and assessment of European structural funds[4]), albeit the creation of an evaluation association and a post-graduate course recently. In Angola, São Tomé e Principe, Guinea-Bissau, and Equatorial Guinea evaluation relevance continues to be underestimated.

 

2. Evaluation demand

Policy decisions in several Lusophone countries are still more driven by values rather than evidence, constraining evaluation demand. In Mozambique, evaluation demand has mainly been engendered by international partners who seek to foster a more results-based budget, while responding to the country´s development and humanitarian needs. In Cabo Verde, development partners, in cooperation with the government, have been supporting endeavors to improve the national planning system at the Direcção Nacional do Plano[5] of the Ministry of Finance and some line Ministries. In Brazil, demand originates in a wide range of actors, such as the government (e.g. via the Brazilian Cooperation Agency), ministries, local governments, foundations and companies. In Portugal, the M&E movement was on the rise before the 2011 financial crisis. Since then, the demand for evaluation demand has been substantively spurred by the European Commission agenda. Other Lusophone countries’ institutions have yet to promote the interest and a greater understanding around M&E, despite demand from donor agencies.

 

3. Evaluation supply

In Lusophone-speaking countries, the field of evaluation is still fragmented, with a shortage of professional organizations dedicated to evaluation, an absence of specialized evaluation journals, and a frail local capacity to conduct advocacy and influence policy-making. The Brazilian M&E supply has been led by academic institutions, such as Fundação Getúlio Vargas (where CLEAR Brazil and Lusophone Africa is located), foundations and consultancy firms. In Cabo Verde, GERA[6] (Group of Reflection and Research on Evaluation) is a civil society initiative that brings together on a voluntary basis independent professionals, public sector experts, Non-Governmental Organisations (NGO) and academia.  In Portugal, Universities and research centers (e.g. NOVAFRICA, ISCTE), together with institutions dealing with European Union (EU) financing mechanisms, small-scale consultancy companies, and independent experts lead the run. In Lusophone Africa and Timor-Leste, there isn´t much of a native trained class of professional evaluators. Supply is mostly donor-driven and executed by international consultants with a limited knowledge of the official language and country context.

 

4. Evaluation capacity

The lack of dedicated academic training and well established and proactive communities of practice has hindered the development of a professional class of evaluators able to promote and disseminate learning and evaluation research. In Brazil, CLEAR Brazil and Lusophone Africa, Rede Brasileira de Monitoramento e Avaliação, and other academic institutions have delivered important trainings in M&E capacity building. In Portugal, Universidade Católica Portuguesa created in 2017 a post-graduate course in social programs and projects´ evaluation. ISCTE has had for long a specialization course in public policy evaluation (Curso de Especialização em Avaliação de Políticas Públicas). Several other Portuguese academic institutions and public training institutions (e.g. Instituto Nacional de Administração) have offered modules/subjects related to evaluation of public policies for over a decade. In Cabo Verde, a seminar on evaluation was organized by the Universidade de Cabo Verde (UniCV) in 2018 and in 2019 an M&E event under the gLOCAL Evaluation Week was organized by the Direcção Nacional do Plano of the Ministry of Finance, the World Bank, and the United Nations. In 2019, a joint initiative by CLEAR Brazil and Lusophone Africa and CLEAR Francophone Africa delivered an M&E training to government officials in Guinea-Bissau.

 

5. Knowledge generation and sharing

There are few strong professional organizations dedicated to evaluation with an overall capacity to conduct advocacy and influence policy-making. Brazil is currently the only country with an active network of M&E professionals, taking the lead in knowledge sharing and dissemination. However, none of these countries have specialized evaluation journals or publications. No partnerships for evaluation have so far been reported in Lusophone countries. One of the main constraints to capacity development pointed out in Lusophone African countries is language. Most training and conferences promoted by development partners and international networks are in English or French, which dramatically lessens the opportunity for wider exchanges and capacity development. Hence, the imperative of a more tailored approach, adapted to Portuguese speaking countries, alongside with the promotion of Results Based-Management practices in government and line Ministries, and the strengthening of statistical systems to obtain better data for SDG monitoring and better policies.

 

What needs to be done

 

Final remarks

 

Despite some initiatives in Portugal and Brazil, where sectoral and national evaluation societies emerged, other Lusophone countries are still lagging behind, seeing mostly small and informal networks fostering short-term training activities. In all countries, there are opportunities to strengthen evaluation supply, with governments playing a more active role in demanding and managing evaluations. Think-tanks and universities hold the potential to enhance evaluation capacity. Donors can also provide opportunities for learning by doing through support within sector-working groups that are country-led.

 

Sharing evaluation experiences among Lusophone and African countries is essential to create and strengthen local evaluation networks, contributing to the development of regional evaluation capacities and to fostering demand for evaluation. This is also vital to ensure that policymakers are aware of the knowledge generated by evaluation and the possibilities of using it to improve policymaking.

 

There are no ready-made solutions for fast-tracking towards the achievement of M&E goals to support the global development agenda. For Lusophone African countries, sound M&E systems linked to planning and budgeting, and capacity development are urgently required. In the short term, a more comprehensive and participatory M&E diagnostic, a strategic plan and a coordinated action to foster partnerships among Lusophone stakeholders are needed to understand which kind of capacity and resources ought to be put in place, who are the champions and what the next steps should be. To achieve this, local, national, regional and supra-regional stakeholders must be engaged in an inclusive exercise.

 

If you are passionate about this subject and wish to join the venture of expanding the Lusophone boundaries in evaluation, stay tuned and contact us at lusophone.evaluation@gmail.com .

 

Estamos juntos!

Mariana Branco 

Elsa Sarmento

Carla Félix