Project Final Evaluation (External Evaluator)

Consultancy Services for EU OASIS Project Final Evaluation (External Evaluator)

BRCiS Consortium

OASIS Project Final Evaluation

Terms of Reference

Background and Context Information

1.1 BRCiS Consortium Overview

Building Resilient Communities in Somalia (BRCiS) is a consortium of national and international organizations – Action Against Hunger (ACF), Concern Worldwide (Concern), GREDO, the International Rescue Committee (IRC), KAALO, Save the Children, and Norwegian Refugee Council (NRC) as lead agency. BRCiS’ objective is to work across the humanitarian-development divide, supporting marginalized communities in disaster-prone, rural Somalia to become more resilient to shocks and stressed, including as a result of climate change. BRCiS approach is contextually adaptive, focused on the specific shocks, needs, and priorities of individual communities. BRCiS was established in 2013 and is now implementing projects funded by multiple humanitarian and development donors in more than ten regions of Somalia[1].

1.2 Programme Background

BRCiS Consortium (including Concern Worldwide, IRC, SCI, NRC, and the Centre for International Forestry Research and World Agroforestry (CIFOR-CRAF) with local partners ZamZam Foundation and SOSTA) is currently implementing the Operationalizing Agro-Sustainability in Somalia (OASIS) project, funded by the European Union (EU). The project objective is to enhance sustainable food production and strengthen food system resilience in Somalia. The project runs from October 2023 to September 2025 and targets 15,000 households (approximately 90,000 individuals) across six districts in Somaliland, Puntland, Galmudug, and Hirshabelle.

OASIS tackles the interconnected challenges of food insecurity, environmental degradation, and climate change by empowering key Somali agricultural market actors including agro-producers (farmers, pastoralists, and agropastoralists) who face challenges like limited access to quality inputs, water infrastructure, and technical knowledge; agro-business service providers (input suppliers, veterinary/animal health centers, and training institutes) struggling with service quality, accessibility, and affordability; natural resource custodians (local authorities and community leaders) responsible for managing environmental resources; and individual consumers, including displaced populations, with low demand for locally produced foods. The project enables these groups to adopt sustainable practices, enhance production capacity, and boost demand for locally produced foods, fostering resilient and sustainable food systems.

The project logic of intervention challenges the assumption of traditional agriculture and food production programming through a resilience-building approach. Improved production capacity alone will have little effect on Somali communities’ resilience to future shocks and stresses if that food is not consumed, nor can improved production and consumption be considered sustainable if they are based on practices that continue to damage or extract communities’ natural resources. Therefore, OASIS adopted the logic of the intervention that is that IF agro-producers and agro-business service providers in target locations increase their production capacity (SO1) AND communities increase their use of context-specific environmental management practices (i.e., climate-responsive and regenerative) (SO2) AND food insecure Somalis increase their consumption of/demand for locally produced foods (SO3) THEN sustainable food production and local food system resilience in Somalia will be enhanced (Impact). Refer to annex 1 for the project key performance indicators (KPIs)

Table 1: Project Implementing Partners, Locations and Targets

1.REGION, District: ADWAL, Baki

BRCiS MEMBER, Local Partner: CWW, SOSTA

Est. final beneficiaries: 1,080 HHs, 6,480 Individuals

2. REGION, District: ADWAL, Boroma

BRCiS MEMBER, Local Partner: CWW, SOSTA

Est. final beneficiaries: 1,407 HHs, 8,442 Individuals

3. REGION, District: BARI, Bosaso

BRCiS MEMBER, Local Partner: SCI

Est. final beneficiaries: 4,780 HHs, 28,680 Individuals

4. REGION, District: GALMADUG, Dhusamareb

BRCiS MEMBER, Local Partner: NRC, CERELPA (SO1 only)

Est. final beneficiaries: 5,000 HHs, 30,000 Individuals

5. REGION, District: HIRSHABELLE, Beladweyne

BRCiS MEMBER, Local Partner: IRC, Zamzam Foundation

Est. final beneficiaries: 2,787 HHs, 16,722 Individuals

6. REGION, District: MARODIJEEX, Gabiley

BRCiS MEMBER, Local Partner: CWW, SOSTA

Est. final beneficiaries: 570 HHs, 3,420 Individuals

Total estimated final beneficiaries

15,624 HHs

93,744 HHs

2. Purpose, Objectives and Scope

2.1 Purpose

This ToR outlines two primary objectives for the OASIS final evaluation

• Endline Assessment of Project Logframe Key performance Indicators (KPIs): The final evaluation will assess the project outcomes against predefined logframe KPIs, see annex 1, through a longitudinal pre-post design (baseline-midline-endline) and comparative analysis of all three phases to systematically measure changes and determine OASIS's contribution to the observed results. Existing project baseline and midline panel datasets will be utilized, with survey data already collected (refer to Annexes 2 and 3 for the finalized baseline and mid-term survey reports).
• Final project evaluation using OECD-DAC Criteria: The final evaluation will employ OECD-DAC criteria to comprehensively assess the project's effectiveness, efficiency, sustainability, and impact during its implementation period (October 2023 - September 2025).

To maximize efficiency and analytical coherence, the project logframe KPI endline assessment and project final evaluation will be merged into one process comprising: (1) standardized questionnaire design, (2) qualitative interview guides, 3) comprehensive data collection, (3) mixed-methods analysis (quantitative and qualitative), and (4) consolidated reporting. The endline survey will function as an integral part of the final evaluation component, delivering essential evidence on project effectiveness and sustainable impact.

2.2 Objectives

2.2.1 Endline Assessment Objectives:

• Assess the project's success in meeting its established logframe KPI targets and overall impact on enhanced sustainable food production and food systems resilience in Somalia. Through comprehensive analysis of project outcome indicators - including improved production capacity, enhanced adoption of context-appropriate environmental practices, and increased local food consumption, it will employ longitudinal comparison of baseline, midline and endline data with disaggregation by gender, age, location and producer type (crop, livestock, dairy or fodder).
• The evaluation will apply rigorous contribution analysis using mixed methods, incorporating participatory discussions with existing baseline case studies (68 in-depth interviews (cases) with project beneficiaries and non-project beneficiaries)
• Generate actionable learning for Future Programming by identifying high-impact interventions suitable for scaling and contextual factors that enabled or constrained success. These insights will specifically inform the design of future initiatives in fragile contexts.

2.2.2 Final Evaluation Objectives

Programme Relevance

The evaluation will assess the alignment between OASIS Program activities/outputs and its intended outcomes, examining how well the intervention design responded to beneficiary needs, national priorities, and EU development objectives. This includes analyzing the consistency between implemented actions and the program's strategic goals for food production and food systems resilience in Somalia.

• Are the activities and outputs of the program consistent with the intended outcomes, and impacts?

Effectiveness and Impact

The final evaluation will document the program's measurable contributions to:

• Strengthening sustainable food production and systems resilience in project target locations.
• Building smallholder farmers and agro-business service providers’ production capacity
• Promoting adoption of context-specific environmental management practices (climate-responsive and regenerative approaches)
• Improving food insecure households’ food security and consumption of locally produced foods.

Using contribution analysis methodology, the evaluation will identify plausible connections between interventions and observed outcomes while accounting for external influencing factors. The review will capture both intended and unintended effects, with particular attention to lasting impacts on target communities/producers’ food security, economic stability, and adaptive capacity.

Efficiency

The final evaluation will examine the cost-effectiveness of program delivery systems, assessing:

• Operational efficiency in activity implementation
• Strategic deployment of human resources
• Systems and processes established for output delivery
• Overall value-for-money consideration

Sustainability

The final evaluation will determine the sustainability of program benefits by analyzing:

• Continued application of improved farming and environmental management practices by smallholder producers
• Retention and replication of knowledge and skills among beneficiaries
• Institutionalization of successful approaches within local systems

2.3 Consultant Scope of Work

The consultant will lead the final evaluation of the OASIS project, ensuring a rigorous and coherent assessment aligned with OECD-DAC criteria, EU standards, and BRCiS requirements. The scope encompasses three key phases:

• Final Evaluation Framework Development & Endline Tool Integration: The consultant will collaborate with the BRCiS Consortium to finalize the final evaluation framework, harmonizing endline data collection with the broader final evaluation. This includes reviewing and refining existing baseline and midterm tools, integrating endline questionnaires and interview guides, and establishing a combined methodology to assess project outcomes against intended objectives.
• Final Evaluation Implementation & Quality Assurance: Upon approval of the final evaluation framework, the consultant will lead all technical aspects of the final evaluation implementation, including: (1) designing final evaluation data collection tools, (2) developing sampling strategies, and (3) establishing data collection and data quality assurance protocols. The consultant will conduct 3) in-person training for OASIS implementing partners on the data collection tools and data quality assurance procedures. BRCiS will manage and conduct all field-level data collection while the consultant may conduct supplementary remote in-depth interviews with project key stakeholders and participants as needed to strengthen the evaluation's depth and perspective. During field data collection managed by BRCiS, the consultant will provide ongoing remote technical support to maintain methodological integrity. This comprehensive approach ensures rigorous adherence to quality standards throughout the data collection process.
• Analysis, Reporting & Dissemination: The consultant will combine final evaluation findings into a consolidated final report, combining endline results with the broader evaluation analysis and the consultant will facilitate a final dissemination workshop in Mogadishu to present the findings to the project stakeholders. The report will provide actionable recommendations to be presented in the dissemination workshop, facilitating stakeholder engagement and learning.

3. Final Evaluation Methodology

The OASIS project employs a rigorous longitudinal mixed-methods approach to assess project effectiveness and contribution to resilient food systems in Somalia. The methodology builds upon three phases of longitudinal data collection (baseline, mid-term, and endline) while incorporating a final project evaluation using OECD-DAC evaluation criteria for comprehensive analysis and reporting.

3.1. OASIS Project Pre-post Longitudinal Evaluation (baseline, mid-term, endline)

OASIS project utilizes a robust mixed-methods longitudinal framework to systematically assess progress against OASIS logframe KPIs, measuring changes in food production and food system resilience outcomes across three phases: baseline (conducted in February 2024), mid-term (conducted in February 2025), and endline (planned under this ToR). The methodology combines quantitative tracking of key indicators including production yields, post-harvest losses, market access, income and adoption of sustainable framing/environmental practices among 1,840[2] randomly sampled households at baseline, with qualitative insights from project stakeholder in-depth interviews. This approach enables both measurable tracking of outcomes and nuanced understanding of implementation challenges and enablers. For detailed information, refer to the baseline report in Annex 2.

For the mid-term, the assessment employed stratified sampling[3] (n=239 at 90% confidence) from the baseline panel respondents focusing on only project participants who were part of the baseline data set to monitor progress of the project KPIs among project participants while maintaining the full baseline panel for comparative analysis. The methodology emphasizes contribution analysis over causal attribution, acknowledging Somalia's complex operating environment and the influence of external factors. Findings are systematically disaggregated by producer type (crop, dairy, poultry, fodder), gender, and location to ensure targeted insights. For more information about mid-term evaluation methodology, refer to the full midline report in Annex 3.

By comparing results across all three phases, the framework provides actionable evidence for adaptive management while demonstrating the project's plausible contribution to resilient food systems. The endline assessment will particularly focus on:

• Longitudinal tracking of logframe KPIs: Measure changes in project outcome indicators across baseline, midline, and endline surveys for both the full baseline sample size (1840 population level baseline panel date set).
• Analyse the project’s contribution by comparing outcomes between the full baseline sample (1,840 households, including both participants and non-participants) and mid-term respondents (239 project participants, a subset of the baseline).
• Strategic disaggregation (production group, gender, location) to inform future programming.

Figure 1: Summarized OASIS M&E plan

Mixed method Endline Data Collection:

• Desk review of project documents, baseline report, mid-term report, and other relevant documentation.
• Endline quantitative surveys with baseline respondents (1,840 households)
• Qualitative methods include:
• In-depth interviews with baseline respondents (68 households)
• Focus group discussions with target communities (28 communities)
• In-depth interviews with market actors (34 market actors).

3.2 Integrated Project Final Evaluation with Endline Questionnaires

The proposed project final evaluation approach is to utilise OECD-DAC evaluation criteria which typically follows a structured approach to assess relevance, effectiveness, impact, efficiency and sustainability and this is integrated with the endline questionnaires. Below are the key steps and focus areas:

• Relevance: Alignment with target communities’ needs and priorities.
• Effectiveness and impact: Achievement of project objectives (e.g., increased food production, improved adoption of sustainable environmental management practices and enhanced consumption of locally produced foods) and long-term effects on food security, livelihoods, and resilience.
• Efficiency: Cost-effectiveness in resource use.
• Sustainability: Continuation of benefits post-funding (institutional, environmental, economic).

Proposed Combined Final Evaluation and Endline Mixed method data collection

• Document Review: Project documents, reports, case studies, baseline & midline reports, any other relevant document
• Final Evaluation Field Data Collection (by BRCiS): Combined endline and final evaluation questionnaires as one tool for each of the below data collection
• Quantitative surveys with baseline respondents (1,840 households)
• Qualitative methods include:
• In-depth interviews with a subset of baseline respondents (68 households)
• Focus group discussions with project target communities (28 communities)
• In-depth interviews with market actors (34 market actors).
• Key informant interviews with government (local authorities)
• Final Evaluation Remote Interviews (by consultant): Additional key informant interviews with market actors, local government authorities, Implementing partners and any other key partner by consultant

4. Key Deliverables

Key deliveries from the consultant include: -

• A comprehensive inception report with proposed and detailed OASIS final evaluation methodology, sampling approach and the integrated endline and final evaluation methodology, data collection tools, roles & responsibilities and work plan.
• Revised endline quantitative and qualitative surveys data collection tools
• Integrated Final evaluation and endline data collection tools.
• Data collection and quality assurance protocols and guidelines
• Training Consortium Members staff on the data collection tools, and data quality assurance protocols.
• Technical support during the data collection process.
• Final report including:
• Final report covering both endline and final evaluation findings (meeting BRCiS/donor EVAL Checklist standards)
• OECD-DAC matrix: Map final evaluation findings into OECD-DAC criteria in the final report.
• Executive summaries
• Presentation decks
• Cleaned raw data and data analysis tables and files
• Learning and actional recommendation briefs from both endline and final evaluation survey
• Dissemination Strategy:
  • Dissemination materials will include, at a minimum, Comprehensive report (covering both endline and final evaluation), slide decks, summary briefs, and these materials should be tailored to suit internal learning and external sharing, depending on the intended audience.
  • Target audiences include EU, CMU, CIFOR-ICRAF, Implementing partners, Somalia government, market actors and project target communities
  • Dissemination Workshops:
    • Community-Level Dissemination led by implementing partners with CMU support; materials will be context-adapted for communities/market actors and will develop by BRCiS.
  • Strategic-Level Dissemination
    • Consultant will facilitate only one in-person 3 days workshops in Somalia for donor, government, and Consortium members. BRCiS will determine the specific audience composition and daily attendance schedule for each workshop, communicating these details to the consultant in a timely manner

Note: All final reports will be delivered in English, with specific formatting details finalized during inception and deliverables will undergo quality review against BRCiS and donor requirements and quality standards. All deliverables will be formatted according to the BRCiS brand charter.

5. Time frame, Key tasks, Deliverables and Roles & Responsibilities

Table 2: Evaluation Phases & Key Activities- Refer to the TOR in Section 4 of the ITB.

6. Management Arrangements and Workplan

Roles & Responsibilities

• Consultant:
• Lead OASIS final evaluation methodology and tools design, data quality assurance and data collection technical backstopping, data analysis and final reporting; ensure BRCiS and EU requirement compliance; train BRCiS staff on these tools and guidelines, conduct key informant interviews with implementing partners and other project stakeholders, facilitate dissemination workshops and deliver all key deliverables in section 4 as per the agreed timeline during inception.
• Consultants must adhere to NRC’s Code of Conduct and data protection protocols. Consultants will sign these documents during the contracting process.
• BRCiS: Oversee contract deliverables, facilitate stakeholder access and dissemination of workshop logistics, approve inception/final reports, and collect all field-based data collection. The supervisor of the consultant is the BRCiS CMU MEL Manager with support from other CMU management teams.
• CIFOR-ICRAF: Support all processes in section 5/table 2, and lead the endline qualitative data cleaning, translation, analysis and preliminary report, review and contribute to the final evaluation report and co-lead the dissemination workshop with BRCiS and consultant
• Implementing Partners/Market actors/Government Local authorities/Community Stakeholders: Participate in FGDs/KIIs and validate preliminary findings

7. Estimated duration of the contract

The engagement is projected to commence in August 2025 and conclude in March 2026.

8. Official travel involved

The selected consultant is required to facilitate the below two sessions in person in Mogadishu:

• Train OASIS Implementing partners on the final evaluation data collection tools and data quality guidelines (~2 days)
• Facilitate OASIS final evaluation dissemination workshops to present the final reports (~ 3days)

For international consultants only, the Consultants will cover all travel and accommodation costs including visa, tax, flight costs, accommodation and meals, and NRC will support in-country transportation, security and administrative support for international firms only. All travel and accommodation and travel-related budget should be included in the financial proposal.

9. Application Procedures and Requirements

The interested consulting firms are expected to provide the following documentation:

1. A cover letter introducing the consultants. The cover letter should introduce the team composition and specify the role to be played by each team member.
2. A technical proposal of no more than 15 pages outlining how to execute the task with a clear proposed final evaluation framework, sampling methodology, and timelines. The proposed methodology should demonstrate a clear understanding of the Terms of Reference (OECD-DAC criteria, OASIS pre-post evaluation methodology, sampling, data collection and analysis strategy/methods). The proposal must demonstrate how the approach meets requirements for contribution-focused evaluation, not impact attribution.
3. Resume of each team member.
4. Evidence of experience conducting similar assignments (Samples of similar work) is required.
5. Proposed budget indicating consultancy fee, logistics cost and all other auxiliary costs in USD. The proposed budget should clearly separate costs/activities for each phase of the consultancy as detailed in Section 5 of this TOR.
6. Considering available project resources, proposals demonstrating cost-effective and streamlined approaches to achieving the objectives will be preferred.

Qualifications or specialized knowledge and/or experience required from the team members and the firm

• Advanced degree (PhD/Master's) in economics, development studies, urban resilience, climate adaptation, or related fields are required.
• At least 7 years of proven experience in designing and implementing Final Project Evaluation Approaches using OECD-DAC Criteria, particularly in dryland/east Africa contexts is required.
• Substantial research work in Food System or a related field with a geographical focus on drylands of the Horn of Africa and preferably in fragile context is required
• Extensive experience both in qualitative and quantitative methods is required.
• Previous experience in evaluating EU funded projects preferred.
• Strong background in contribution analysis is an asset.
• Previous experience in similar assignments in Somalia is an asset.
• Only consulting firms are eligible to apply; individual consultants will not be considered.

Annexes

Annex 1: OASIS Project Logframe KPIs- Refer to the TOR in Section 4 of the ITB.

[1] https://www.nrc.no/brcis

[2] There were no project participants/beneficiary’s registry at the time of baseline data collection hence a random walk sampling approach has been utilized. Approximately 28% of the baseline respondents were direct project participants at baseline. Qualitative data from interviews and discussions with food system actors (n=34), farm-level households (n=70), and community members (n=28) complemented the quantitative findings. For detailed information about the baseline methodology

[3] The mid-term progress monitoring purpose conducted in February 2025 and the purpose was to track a subset of the baseline respondents who were part of the project in the past year, enabling systematic analysis of key performance indicator (KPI) trends and measurable changes between the baseline and current the mid-term progress monitoring survey for accurate progress reporting. A sample size of 239 respondents was calculated using stratified sampling at a 90% confidence level, based on the total number of targeted producers, 1990 farmers

How to apply

Firms that meet the above requirements are invited to download the complete tender documents free of charge from the Digital Procurement System (DPS). Eligible applicants must submit a comprehensive administrative, technical, and financial proposal via the NRC Digital Procurement System (DPS) not later than 18th August 2025 at 11:59PM.

https://eaydps.nrc.no/tender/details/9f8dd230-5ccb-494e-abf0-a9335d5fa0f3

If you have any technical difficulties in accessing the tender documents, please contact the following email address: so.procurement@nrc.no for assistance, considering that NRC will not share the bids via email as all bidders must Register and download the tender documents from the Digital Tendering System.