Andy Rowe and Debbie DeLancey
A recent report released by the Canadian Evaluation Society (CES) assesses readiness of evaluation in Canada to address sustainability. It finds that evaluations that consider sustainability and the natural system more generally are infrequent and capacity is limited.
The Assessment[1]
The review considered the readiness of evaluation in Canada to address sustainability in four areas: the Canadian Federal Government’s evaluation efforts, the work of philanthropic and environmental non-government organizations focusing on natural systems, international work being conducted by Canadian evaluators, and the intellectual infrastructure addressing sustainability-ready evaluation.
With respect to the Canadian Federal Government, the assessment focused on evaluations conducted under the well-developed government-wide federal Canadian evaluation structure. Under the National Evaluation Policy all policies and programs are evaluated at least once in a five-year cycle, and the sustainability review examined all evaluations completed between 2016 and 2018 by the evaluation units of Departments whose programmatic efforts were judged most likely to raise sustainability issues. A total of 77 evaluations conducted between 2016 and 2018 were carefully reviewed – sustainability was considered in few. Departments whose remits addressed resources (e.g. Fisheries and Oceans Canada, Natural Resources and Climate Canada) only considered human system such as employment and sector development. Sustainability and natural resources are rarely considered by federal evaluations in Canada, despite stated policy.
The second major element in the stocktaking was a review of the intellectual infrastructure of evaluation in Canada to address sustainability. This included published materials in journals and on websites, conference presentations and grey literature . (For resource considerations this search covered only Canada and the US, which excludes important European and other sources and only looked at the period of 2017-2019). The findings were equally sobering: for example, under 4% of articles published in four leading North American evaluation journals[2] considered natural resource matters, and only a few of these considered sustainability. Presentations at the Canadian and American national evaluation organisations painted a similar picture. While treated less systematically, a review of philanthropic and non-government commissioners of evaluations revealed that they primarily addressed human system matters. The only exceptions were natural system-focused organisations such as conservation philanthropies, which focused on the natural system almost exclusively.
The main message from the CES stocktaking is that evaluation in Canada has not considered the natural system and sustainability worthy of consideration. Further, the intellectual infrastructure in Canada and the US for evaluation of the natural system is very limited.
It is notable that the UN Evaluation Group (UNEG) has recently concluded a similar stocktaking with similar findings to the CES stocktaking. The UNEG report concluded the over-arching need emerging from documentary analysis and survey responses of UNEG member Agencies is for a comprehensive document providing advice on how to evaluate the interactions among social and environmental considerations within the framework of UN activities in support the SDGs[3]. What we observed with the CES stocktaking is reflected in evaluation globally.
What gave rise to the sustainability-readiness stocktaking? How is CES addressing sustainability?
A motion from the floor at the 2018 Annual General Meeting of CES directed the organisation to review the readiness of evaluation in Canada to address sustainability. A Sustainability Working Group was established shortly thereafter and undertook to assess the sustainability practices of CES, and to conduct the sustainability-ready assessment[4]. The CES Board released a 2019-2024 Strategic Plan that included as a principle the following statement: We take a leader role in sustainability, building opportunities to align work in diverse sectors in support of sustainability, and incorporating increasingly sustainable practices as an organization. The stocktaking report was completed December 2020 and received unanimously by the Board in February. Sustainability is one of two substantive pillars in the new strategic plan, the other being Diversity, Equity and Inclusion. Member Services is a third pillar. The Board has established a national Diversity, Equity, Inclusion and Environmental Sustainability committee to plan how CES will address these pillars. The scope for further actions is likely to be quite broad including professional development, standards, credentialing, and advocacy as well as the operations of CES itself.
Going forward
CES is the first national evaluation association to formally assess the readiness of its evaluation function to address sustainability, and the stocktaking was the first globally to assess actual evaluation practice and resources. In doing so it has developed approaches that have proved fit for purpose and can easily be adopted by other national evaluation associations. Important initiatives have emerged that will significantly aid efforts to systematically incorporate sustainability into evaluation, for example Footprint Evaluation (https://www.betterevaluation.org/en/themes/footprint_evaluation) and Blue Marble evaluation (https://bluemarbleeval.org/) among others.
[1] The stocktaking was overseen by Andy Rowe and Debbie DeLancey and was undertaken on a pro bono basis by four leading Canadian evaluation firms: Baastel, Goss Gilroy, Prairie Research and Universalia.
[2] Canadian Journal of Program Evaluation, New Directions in Evaluation, American Journal of Evaluation, and Evaluation and Program Planning.
[3] Todd, D. (2020). UNEG working group on integrating environmental and social impact into evaluations, Volume One Main Report. United Nations Evaluation Group.
[4] SWG members have included Andrealisa Belzer (Chair), Matt Jacques (SWG National Board Liaison), Andy Rowe, Benoit Gauthier (2018-19) and Debbie DeLancey (2019-20).