USAID-supported Youth for Peace (Y4P) Activity

Endline Evaluation Terms of Reference (ToR) for Consultancy

This Term of Reference (TOR) provides the description and the work required to conduct an endline evaluation of USAID-supported Youth for Peace(Y4P) Activity with Agreement No: 72066921CA00004, implemented in Montserrado, Bong, and Nimba Counties from August 6, 2021, to August 5, 2024. The work required by this TOR for the endline evaluation includes the design of the final evaluation approach/methodology, implementation, and reporting of findings. Thus, the aim among other uses of the endline evaluation is to provide impact and learning for future USAID Activity designs based on the assessment of the implementation results achieved as targeted and for reporting on the annual USAID Development Information Solution (DIS) platform.

This TOR consists of the following sections: Introduction, Background; Justification; Objective; Evaluation Questions; Methodology; Deliverables; Qualifications; Roles and Responsibility; Timeline; Payment schedule; and Submission Guidelines.

1. Introduction

CRS – Liberia Youth for Peace (Y4P) Activity commissions an external evaluation to assess the implemented activities and the interventions, delivered to the selected communities in Bong, Nimba, and Montserrado Counties, led to the achievement of the planned results as well as to the Y4P Activity objectives. Given that the Activity is in its final year, recommendations from the evaluator(s) are expected to improve the quality of future projects.

The study will be conducted beginning June 03 – July 15, 2024, which is the final month of the implementation period for the Y4P Activity contract period.

The evaluation processes are split into three phases:

1. The preparation,
2. Field data collection and synthesis, community-level data validation, and,
3. Final reporting phase.

The preparation phase consists of inception meetings, review of Activity results and indicators, activity implementation, annual reports, development of evaluation framework/tools, sampling, and methodology and training of enumerators.

The field phase involves the collection of household data across the 24 target communities. In Bong, data will be collected in the following communities, namely Gold Camp, Palala, Bong Mines, Gbaota, Gbartala, Gbarnga City, Totota, and Sgt. Kollie Town. For Nimba, data will be collected from Saclepea, Bahn, Karnplay, Ganta, Sanniqullie, Yekepa, Viepa, Sehyi Kimpa. and in Montserrado, endline data will be collected from Central Logan town, Bardnesville-Patience Shop, Duport Road, ELWA-King Gray, Caldwell New Georgia, New Kru Town, Parker Pain, and Goba Town-Bahr. Data cleansing and analysis will be conducted at the end of the data collection to allow the team to provide a summary of the evaluation key findings. Said information will be used to inform the Activity stakeholders about the impact-level success of the project.

The final reporting and national validation workshop are a visual presentation of data validated, incorporating feedback, development of a draft report for program Manager review, and finalization of the report. The final report presentation marks the end of the endline evaluation survey phase.

1. Background

Since the silencing of the guns in 2003, Liberians have witnessed three successive democratic elections, but they have not experienced the dividends of peace. The processes of post-conflict healing and reconciliation did not run deep enough as the Truth and Reconciliation process was truncated at the national level. As a result, many Liberians are still living with the memories of hurt, pain, loss, and in many cases, hatred in their hearts. Beyond that, the structural and systemic causes of the war – inequitable distribution of development and economic resources; limited access to basic needs and services outside Monrovia; perceived high rates of corruption in government; persistent rivalries and conflicts within and between ethnic groups; and the high level of civic distrust of state officials have not been addressed substantively and justly. Unemployment and underemployment of large numbers of youth who lost out on education and job-ready training due to the war fuel a sense of betrayal, especially among those who fought in the war in the hope of a better life. The foregoing has frequently led to violent confrontations between communities and/or with the police.

To address these challenges, Catholic Relief Services (CRS) Liberia partnered with the National Commission of Catholic, Justice, Peace, and Caritas (NCJPC), Messengers of Peace (MoP), and the Center for Justice and Peace Studies (CJPS), to implement the Youth for Peace (Y4P) Activity, which is funded by the United States Agency for International Development. The Y4P Activity is a 36-month initiative designed to work with conflict-affected and marginalized youth in Montserrado, Bong, and Nimba Counties to contribute to positive peace in Liberia.

CRS led the technical and organizational capacity development and ensured quality monitoring, evaluation, reporting, and learning. In responding to the structural and systemic causes of conflict, the Youth for Peace (Y4P) Activity used the below strategic objectives and intermediate results. The Y4P activities targeted 2,655 (1,440 marginalized youth, 1080 FOMTUL members, and 135 Local Authorities) in Bong, Montserrado, and Nimba Counties.

GOAL: Conflict-affected and marginalized youth in Montserrado, Bong and Nimba counties contribute to positive peace in Liberia

Strategic Objective 1: Trust between youth and community members strengthened.

• Intermediate Results 1.1: Youth develop resilience and reconciliation skills.
• Intermediate Results 1.2: Youth promote peace in their communities.
• Intermediate Results 1.3: Youth engage in productive economic activities.

Strategic Objective 2: Relationships between youth and locally elected and appointed officials improved.

• Intermediate Results 2.1: Youth acquire skills to hold elected and appointed officials accountable.
• Intermediate Results 2.2: Multi-media (theater and radio) serve to bridge youth with elected and appointed officials.

Justification for Endline Evaluation

This endline evaluation is to be carried out as one of the required evaluation activities in the approved project Activity Monitoring, Evaluation, and Learning (AMEL) plan, for assessing the relevance, effectiveness, efficiency, coherence, and sustainability of the project results and approaches. Similarly, the CRS’ MEAL Policies and Procedures, require a conduct of a utilization-focused evaluation that satisfies the information needs of the donor, government, as well as CRS and its partners. This information needs, amongst others, would include providing accountability for the USG investments, identifying lessons learned (to improve future interventions), and promising and sustainable practices and results that can be scaled up by CRS and/or the Government.

Overall Goal of the Endline Evaluation

The overall objective of this evaluation is to assess, as systematically and objectively as possible, the above-described Activity, its design, implementation, and results. The consultant(s) should make evidence-based recommendations where appropriate regarding changes to the project design, scope and implementation, management approach, and methodologies to improve the performance and delivery of similar or future projects. The evaluation will focus also on assessing the cost-effectiveness of the Activity, and review of structures, processes, and systems established and gaps during the implementation period.

Objectives of the Endline Evaluation

The end-line evaluation will assess the impact of service delivery as outlined in the detailed implementation plan and technical narrative and will measure progress against stated goals and objectives, including the effectiveness, efficiency, and timeliness of the Activity interventions in achieving targets against baseline value. The evaluation will review the results frameworks with all project documents and critical assumptions and consider the implementing environment, including enablers and constraints, to communicate the impact of the Activity to meet its stated goal and objectives and achieve sustainability. Specifically, the evaluation will provide the following:

1. Assess the relevance of the Y4P Activity priorities, objectives, and implementation plan to the needs of the participants (as defined in the Activity inception documents).
2. Assess the effectiveness of the Activity intervention in achieving its expected outcomes (planned versus achieved)
3. Determine the efficiency of the Activity implementation and operation in achieving the output deliverables from the inputs and budgets deployed by the project.
4. Appraise the coherence of project approaches, management, and coordination processes with the actions, structures, and processes of the Government and other actors within the Sector and geographical spheres of the Water for Agriculture activity.
5. Identify evidence of programming strengths, weaknesses, emerging opportunities, lessons learned, and sustainability of project gains in the community.

In addition, the evaluation aims to establish: 1. The extent of progress on the Activity implementation, and intervention 2. Implementation experiences, mostly from the study team, and implementing partners 3. Outcome and/or influence of the intervention on the effectiveness of working with marginalized youth (zogos/es), commercial bike riders, and political parties’ youth on conflict prevention and management and the current interplay locally elected and appointed officials of government and the youth, 4. Sustainability of the intervention 5. Understand the contextual factors that interface with the outcome of the intervention.

Indicative Evaluation Criteria and Questions

The Consultant will work with the CRS Team to co-create an evaluation matrix (using this preferred template) that will inform field data collection, management, and reporting for the end-line survey. The following is an indicative list of the evaluation questions to guide the design of the end-line study. The successful consultant(s) can review, redefine, comment, or adjust the evaluation questions in their proposal where necessary to inform further discussions with the CRS Team.

Methodological approach

The end-line evaluation will utilize the same methods as the baseline. It will consider qualitative and quantitative approaches in the collection of data.

A survey will be done with a random sampling of 480 households in all 24 towns of the Y4P Activity communities located in Bong, Nimba, and Montserrado Counties. Enumerators will be trained to collect data under the supervision of the Consultant with support from the Program Manager, and CRS MEAL Technical Associate. The end-line evaluation will apply a mix of quantitative and qualitative methods to assess the effectiveness of strategies and implementation interventions and provide recommendations to further improve the primary areas of conflict prevention, resolution mechanisms, and social cohesion.

6 focus group discussions (FGDs) will be facilitated by the research team.

18 key informant interviews (KIIs) will be held with various key stakeholder groups including women, youth, elders, and members of civil society and community-based groups. These will also be done with one town chief per district (informal leader), and the district commissioner of each district (formal leader). These interviews will be used to (i) understand the challenges and obstacles the leaders had in resolving conflict during the project life, (ii) ways that the project assisted them in overcoming these obstacles, (iii) whether the leaders were incorporated into the startup of the project to enhance buy-in, and (iv) triangulating the information received from community members and groups.

The results of the evaluation will be presented to the Activity management team in a reflection meeting at the CRS Monrovia office with the attendance of the Donor community, relevant stakeholders, Government Officials, Local Partners, and CRS Staff. The final report will be shared by e-mail with the CRS Team. Results will be discussed and integrated into the project’s monitoring and evaluation system.

1. Period of the End-line Evaluation

The Y4P Activity study is scheduled to be conducted between June 3rd and July 16th, 2024, encompassing the 24 targeted communities. The study will follow a well-defined timeline:

• June 3rd – 7th: Inception Meeting – This period will involve a discussion on the study methodology, review of study tools, documents required from CRS to support the research team, etc.
• June 10th – 12th: Enumerator Training & Field Testing – Selected enumerators will undergo comprehensive training followed by practical field testing to ensure competency.
• June 13th: Field Deployment – Enumerators will travel to the designated field locations in Bong and Nimba Counties.
• June 14th – 27th: Data Collection Phase – Intensive data collection will be undertaken within the targeted communities.
• June 28th – July 15th: Data Processing & Reporting – Collected data will undergo thorough cleaning, analysis, and validation, and be used to compile a comprehensive report.

Endline Evaluation Design

The end-line evaluation will be conducted with the utmost transparency, ensuring comprehensive stakeholder engagement. This includes consultations with a broad range of relevant stakeholders, encompassing community leaders, local authorities, and religious leaders, as appropriate. To ensure the findings’ robustness, disaggregation of evaluation data will be conducted by age, gender, and, where necessary, factors such as socio-economic status and disability.

A detailed methodology will be outlined, comparing the end-line results with the established baseline data. This comparison will assess the program’s overall impact and effectiveness. The evaluation methodology will adhere to the established guidelines set forth by both USAID’s Evaluation Policy and CRS’ Monitoring, Evaluation, Accountability, and Learning (MEAL) Policies and Procedures. The evaluation will employ a mixed-methods approach, incorporating a thorough literature and desk review alongside field research. This field research will utilize a combination of quantitative and qualitative data collection methods to provide a holistic view of the program’s effects.

To ensure a cohesive understanding of the evaluation approach, data collection, analysis, and reporting processes, the consultant(s) will collaborate closely with the CRS MEAL and Programming teams. This collaborative approach will be facilitated through inception meetings, the joint development of an Evaluation Matrix, data collection tools, and an evidence assessment framework. Regular progress checks will be conducted throughout the evaluation study to maintain clear communication and alignment.

The literature and desk review will be the consultancy team’s first step to properly understand the Y4P Activity. The CRS team will provide the:

• Activity result framework/ IPTT
• Baseline report
• Project description (proposal),
• Detailed implementation plan,
• Approved AMEL Plan,
• Learning questions,
• All progress reports submitted to the donor.
• CRS’ MEAL Policies and Procedures
• Other publications by stakeholders

A robust data collection strategy will be employed to gather in-depth information for the evaluation. This strategy will encompass a variety of methodologies, potentially including structured interviews, focus group discussions, and key informant interviews. Additionally, data will be collected from organizations supported by the Activity, along with groups and individuals who have directly and indirectly benefited from the project. CRS staff and implementing partners will also be included in the data collection process. To gain further context and enrich the data, direct observation through visits to supported community groups will be conducted.

The consultant(s) will be responsible for proposing the most appropriate mix of these data collection approaches. This selection will be tailored to effectively address the established evaluation questions and ensure a comprehensive assessment.

The data collection process must ensure the following:

• Respondents’ privacy and confidentiality are protected.
• Informed consent and/or assets are obtained from all respondents in accordance with extant laws, ethics, and guidelines for human subject research standard research ethics.
• Minimize risk and ‘do no harm’ to the Activity participants and respondents of the evaluation.
• Only required data/information will be sought from the respondents and analysis restricted to the purpose of this evaluation.
• Uphold the highest standard of research/evaluation participants’ protection approaches and duty of care.

Presentation of the report and documentation of conclusions and recommendations Following data collection and analysis, a collaborative validation process will be undertaken to ensure the comprehensiveness and credibility of the evaluation’s key findings and recommendations. This process will involve the consultant(s) working closely with the Program Manager to jointly validate these aspects of the evaluation. To foster continuous improvement and knowledge sharing, a reflection meeting will be convened to discuss the lessons learned throughout the evaluation process.

The report will be structured as follows:

• A table of contents
• Executive summary: this summary will address the main conclusions, with a maximum of two paragraphs describing the program, a summary of objectives and expected results; recommendations for the design/implementation of future programs; key findings (1. relevance and appropriateness of project design, 2. Effectiveness of the Activity intervention strategies, 3. Activity efficiency and 4. Sustainability) ·
• Background to Evaluation: This is inclusive of the Theory of Change, Strategies, and key activities.
• Evaluation Design and Conduct: Purpose and the scope of the evaluation, evaluation approach and methods
• Evaluation Findings
• Reflection on Findings
• Key lessons and good practices
• Recommendations
• Appendices
• Notes and References

Evaluation roles and responsibilities:

Consultancy Main Activities

Evaluation timeline and deliverables:

Evaluation Timelines

1. Consultancy Deliverables

Required Qualifications

Proposals will be accepted from independent consultants/firms with verified previous experiences in conducting evaluations in the field of peacebuilding with a focus on youth programming in post-conflict or conflict-affected individuals. The consultants should possess the following combination of skills and expertise:

For individuals

• Postgraduate degree in peace and conflict studies, social sciences, development studies, etc., and with formal research skills. An MPhil. or PhD holder will be highly considered.
• At least 5 years of advanced experience in conducting evaluations of complex program design with a clear understanding of various social research methodologies.
• Demonstrated research and analytical report writing skills with sound experience in participatory review and evaluation methodologies.
• Experience in multi-sector and partnership approaches to peacebuilding, social cohesion, and youth empowerment programming.
• Ability to conduct interviews and discussions in English and local language desirable.
• Ability to analyze and synthesize data from different sources relating to the scope of the evaluation.
• Ability to clean, analyze and synthesize data using survey data management packages such as CommCare, CSPro and Excel as well as SPSS, STATA, SAS, and or other appropriate software to analyze and report large datasets.
• Good eye for detail, adherence to logic, and capacity for inductive reasoning
• Strong presentation, facilitating, communication and team working skills.
• Strong computer skills and knowledge of ICT4D.
• Good interpersonal skills, including the ability to conduct discussions with a diversity of people ranging from senior management to project participants.
• Willingness and ability to travel to the different Y4P Activity communities in the country.

For Consultancy Firm

• At least two lead researchers with Master’s degree in Social Sciences, Economics, or at least ten years of relevant experience including serving as lead researcher on Youth peacebuilding and conflict prevention
• Knowledge and experience in using Statistical Package for Social Sciences (SPSS) software for data analysis.
• Ability to use data collection tools such as CommCare for data collection and analysis and Power BI for data visualization.
• Ability to deliver expected outputs on time.
• Registered firm with at least five years of operation in Liberia
• Possess experience in implementing at least three similar research projects for other INGOs in Liberia. References must be provided.
• Possess valid business registration and tax clearance.

Assigned Roles

Project Team

The Activity team will be responsible for providing the existing documents related to the project and logistics support to the consultant during the field data collection, and final evaluation presentation to CRS.

Evaluation Consultant

The role of the evaluation consultant is to work closely with the project team to develop the evaluation design; and its implementation – outlining the methodology, and key evaluation questions, identifying appropriate evaluation tools, developing the data collection instruments, carrying out data collection, data analysis and writing the draft and final evaluation report. The consultant will present the evaluation plan and findings to the project team. S/he also should adhere to USAID evaluation principles and standards for conducting project evaluation. The consultant is expected to make a formal presentation of the report to the project team.

Supervision

CRS MEAL team will be involved in all the phases of the final evaluation survey and will take a supervisory role in ensuring quality and data integrity. Specifically, the team will be responsible for recruiting the survey team (enumerators) for data collection and other supervisory responsibilities Payment Schedule

Consultancy Terms of Payment

Payment will be made to the consultant on a tranche basis. The first payment tranche is 50% of the agreed cost for the consultancy which covers field preparation, data collection and analysis. The final tranche of 50% will be paid upon the Y4P Activity (USAID) donor’s approval of the final evaluation report submitted.

Timeline

The evaluation is expected to be finalized within a period of not more than 31 working days, commencing June 3, 2024. The consultant will work with the team to assess the Lessons learned and recommendations of the Y4P Activity.

Application Submission Procedure

Interested individuals or consultancy firms are required to submit the following documentation:

• Cover Letter (Maximum one page)
• Technical and financial proposals with a focus on addressing the assignment’s purpose and objectives, methodology to be used, key selection criteria, sample size from the study population and targeted age groups and draft data collection tools and interview guides.
• Initial work plan based on the methodology outlined.
• Company or individual profile or CV including at least 3 references from previous INGOs.
• Detailed budget breakdown.
• Business registration and tax clearance for the company.
• Report on similar research work conducted in the past.

The deadline for submission of both technical and financial proposals is 24^th of May 2024. Proposal should be submitted to liberiaprocurement@crs.org with subject: USAID-Supported Youth for Peace (Y4P) Activity Endline Evaluation Application. Please note that application will be reviewed on a rolling basis.