Five lessons from our Transparent, Reproducible, and Ethical Evidence (TREE) reviews | 3ie
The International Initiative for Impact Evaluation (3ie) is working on developing stronger systems for producing Transparent, Reproducible, and Ethical Evidence (TREE). 3ie are refining this process to consider ethical questions throughout the evaluation life cycle and document risk assessment and management strategies. The TREE Review Framework aims to establish ethical standards, integrate TREE best practices into the workflow, and establish independent oversight of risks to produce credible, meaningful, ethical evidence. 3ie has engaged with more than 15 study team members across five TREE Reviews in four countries to pilot test and implement this process since December 2021. Lessons learned from this diverse experience include the need for clear and consistent documentation of assessments and mitigation strategies, considering TREE practices in evaluation planning, reflecting on team composition, roles, and responsibilities, understanding who is accountable for risks, and using TREE Review to support more meaningful engagement with Institutional Review Boards (IRBs). 3ie welcomes input and feedback on the TREE process and tool.
Developing LateriteAI – technology to automate tedious research tasks | ITAD
Laterite, a data, research, and analytics firm, has developed LateriteAI, an online service that utilises large language models to automate tedious research tasks such as extracting survey data, checking bias, and coding qualitative data. The suite of tools aims to accelerate tasks and foster collaboration between researchers, saving time and improving research quality. However, while large language models present tremendous opportunities to streamline research tasks and enhance efficiency, they are still unreliable for more nuanced tasks that require intricate judgement, emphasising the importance of human expertise. Striking a balance between automation and human expertise is crucial to maximising the opportunities presented by new technologies. Developing evaluation frameworks, benchmarks, and methodologies is essential to align models with research intent and handle nuanced tasks effectively. Addressing data privacy and security concerns is also crucial to maintain trust and compliance with evolving data protection regulations. Researchers can contribute their own applications to LateriteAI, benefiting from the collective ingenuity of their peers.
UNFPA Evaluation Strategy 2022-2025
The United Nations Population Fund (UNFPA) has highlighted the need for a strong evaluation function to support accountability, learning, and adaptation in its second strategic plan for 2022-2025. A robust evaluation function is considered indispensable for improving programming effectiveness, strengthening the evidence base for decision-making, and ensuring accountability for learning for results. UNFPA’s strategic plan commits to maximizing organizational learning through evaluation designs that prioritize human rights-based, gender-responsive, and disability-inclusive evaluations. The COVID-19 pandemic has made evidence from evaluations even more essential to catalyze decisions towards building more equitable and resilient societies. The Evaluation Office has realigned its work to better support the implementation of the strategic plan and to deliver timely and relevant evaluation evidence. The strategy aims to better position the UNFPA evaluation function relative to other functions, strengthen internal evaluation capacity development, and contribute to national evaluation capacity development while meeting accountability standards and learning needs of UNFPA.
Agriculture is crucial for enhancing living standards, food security, and nutrition in sub-Saharan Africa and South Asia, where 50% of the labor force is engaged in agricultural activities. To increase productivity and incomes, the Digital Agricultural Innovations and Services Initiative (DAISI) evaluates programs that provide bundled digital agricultural solutions for small-scale producers. DAISI focuses on six main research themes and three cross-cutting themes: empowering women farmers, engaging youth in agriculture, and increasing climate change resilience.
DAISI offers funding, mentorship, and training for researchers through its Regional Scholars program. Seven projects have been awarded funding, covering diverse topics such as climate-resilient agriculture, resource use and agriculture, and improving outcomes for women farmers. Examples include increasing production of stress-tolerant crops, piloting drone delivery of livestock vaccines, and evaluating gender-responsive agricultural apps.
As digitalization continues to expand, DAISI aims to align digital solutions with small-scale producers’ priorities, promoting cost-effective and policy-relevant interventions across sub-Saharan Africa and South Asia.