USAID-supported Youth for Peace (Y4P) Activity
Endline Evaluation Terms of Reference (ToR) for Consultancy
This Term of Reference (TOR) provides the description and the work required to conduct an endline evaluation of USAID-supported Youth for Peace(Y4P) Activity with Agreement No: 72066921CA00004, implemented in Montserrado, Bong, and Nimba Counties from August 6, 2021, to August 5, 2024. The work required by this TOR for the endline evaluation includes the design of the final evaluation approach/methodology, implementation, and reporting of findings. Thus, the aim among other uses of the endline evaluation is to provide impact and learning for future USAID Activity designs based on the assessment of the implementation results achieved as targeted and for reporting on the annual USAID Development Information Solution (DIS) platform.
This TOR consists of the following sections: Introduction, Background; Justification; Objective; Evaluation Questions; Methodology; Deliverables; Qualifications; Roles and Responsibility; Timeline; Payment schedule; and Submission Guidelines.
CRS – Liberia Youth for Peace (Y4P) Activity commissions an external evaluation to assess the implemented activities and the interventions, delivered to the selected communities in Bong, Nimba, and Montserrado Counties, led to the achievement of the planned results as well as to the Y4P Activity objectives. Given that the Activity is in its final year, recommendations from the evaluator(s) are expected to improve the quality of future projects.
The study will be conducted beginning June 03 – July 15, 2024, which is the final month of the implementation period for the Y4P Activity contract period.
The evaluation processes are split into three phases:
The preparation phase consists of inception meetings, review of Activity results and indicators, activity implementation, annual reports, development of evaluation framework/tools, sampling, and methodology and training of enumerators.
The field phase involves the collection of household data across the 24 target communities. In Bong, data will be collected in the following communities, namely Gold Camp, Palala, Bong Mines, Gbaota, Gbartala, Gbarnga City, Totota, and Sgt. Kollie Town. For Nimba, data will be collected from Saclepea, Bahn, Karnplay, Ganta, Sanniqullie, Yekepa, Viepa, Sehyi Kimpa. and in Montserrado, endline data will be collected from Central Logan town, Bardnesville-Patience Shop, Duport Road, ELWA-King Gray, Caldwell New Georgia, New Kru Town, Parker Pain, and Goba Town-Bahr. Data cleansing and analysis will be conducted at the end of the data collection to allow the team to provide a summary of the evaluation key findings. Said information will be used to inform the Activity stakeholders about the impact-level success of the project.
The final reporting and national validation workshop are a visual presentation of data validated, incorporating feedback, development of a draft report for program Manager review, and finalization of the report. The final report presentation marks the end of the endline evaluation survey phase.
Since the silencing of the guns in 2003, Liberians have witnessed three successive democratic elections, but they have not experienced the dividends of peace. The processes of post-conflict healing and reconciliation did not run deep enough as the Truth and Reconciliation process was truncated at the national level. As a result, many Liberians are still living with the memories of hurt, pain, loss, and in many cases, hatred in their hearts. Beyond that, the structural and systemic causes of the war – inequitable distribution of development and economic resources; limited access to basic needs and services outside Monrovia; perceived high rates of corruption in government; persistent rivalries and conflicts within and between ethnic groups; and the high level of civic distrust of state officials have not been addressed substantively and justly. Unemployment and underemployment of large numbers of youth who lost out on education and job-ready training due to the war fuel a sense of betrayal, especially among those who fought in the war in the hope of a better life. The foregoing has frequently led to violent confrontations between communities and/or with the police.
To address these challenges, Catholic Relief Services (CRS) Liberia partnered with the National Commission of Catholic, Justice, Peace, and Caritas (NCJPC), Messengers of Peace (MoP), and the Center for Justice and Peace Studies (CJPS), to implement the Youth for Peace (Y4P) Activity, which is funded by the United States Agency for International Development. The Y4P Activity is a 36-month initiative designed to work with conflict-affected and marginalized youth in Montserrado, Bong, and Nimba Counties to contribute to positive peace in Liberia.
CRS led the technical and organizational capacity development and ensured quality monitoring, evaluation, reporting, and learning. In responding to the structural and systemic causes of conflict, the Youth for Peace (Y4P) Activity used the below strategic objectives and intermediate results. The Y4P activities targeted 2,655 (1,440 marginalized youth, 1080 FOMTUL members, and 135 Local Authorities) in Bong, Montserrado, and Nimba Counties.
GOAL: Conflict-affected and marginalized youth in Montserrado, Bong and Nimba counties contribute to positive peace in Liberia
Strategic Objective 1: Trust between youth and community members strengthened.
Strategic Objective 2: Relationships between youth and locally elected and appointed officials improved.
Justification for Endline Evaluation
This endline evaluation is to be carried out as one of the required evaluation activities in the approved project Activity Monitoring, Evaluation, and Learning (AMEL) plan, for assessing the relevance, effectiveness, efficiency, coherence, and sustainability of the project results and approaches. Similarly, the CRS’ MEAL Policies and Procedures, require a conduct of a utilization-focused evaluation that satisfies the information needs of the donor, government, as well as CRS and its partners. This information needs, amongst others, would include providing accountability for the USG investments, identifying lessons learned (to improve future interventions), and promising and sustainable practices and results that can be scaled up by CRS and/or the Government.
Overall Goal of the Endline Evaluation
The overall objective of this evaluation is to assess, as systematically and objectively as possible, the above-described Activity, its design, implementation, and results. The consultant(s) should make evidence-based recommendations where appropriate regarding changes to the project design, scope and implementation, management approach, and methodologies to improve the performance and delivery of similar or future projects. The evaluation will focus also on assessing the cost-effectiveness of the Activity, and review of structures, processes, and systems established and gaps during the implementation period.
Objectives of the Endline Evaluation
The end-line evaluation will assess the impact of service delivery as outlined in the detailed implementation plan and technical narrative and will measure progress against stated goals and objectives, including the effectiveness, efficiency, and timeliness of the Activity interventions in achieving targets against baseline value. The evaluation will review the results frameworks with all project documents and critical assumptions and consider the implementing environment, including enablers and constraints, to communicate the impact of the Activity to meet its stated goal and objectives and achieve sustainability. Specifically, the evaluation will provide the following:
In addition, the evaluation aims to establish: 1. The extent of progress on the Activity implementation, and intervention 2. Implementation experiences, mostly from the study team, and implementing partners 3. Outcome and/or influence of the intervention on the effectiveness of working with marginalized youth (zogos/es), commercial bike riders, and political parties’ youth on conflict prevention and management and the current interplay locally elected and appointed officials of government and the youth, 4. Sustainability of the intervention 5. Understand the contextual factors that interface with the outcome of the intervention.
Indicative Evaluation Criteria and Questions
The Consultant will work with the CRS Team to co-create an evaluation matrix (using this preferred template) that will inform field data collection, management, and reporting for the end-line survey. The following is an indicative list of the evaluation questions to guide the design of the end-line study. The successful consultant(s) can review, redefine, comment, or adjust the evaluation questions in their proposal where necessary to inform further discussions with the CRS Team.
Relevance | Effectiveness | Efficiency | Impact | Coherence | Sustainability |
1. How relevant are the interventions to the context and USAID development objectives?
2. How relevant are the interventions to the needs of the Activity participants?
3. How has the project changed over time to the changing context?
4. Were the activities sensitive to local needs and nuances of the Activity communities or participants’ groups? If not, what could have been done differently?
5. To what extent was the project gender and conflict-sensitive?
| 6. To what extent were the objectives of the project and the yearly benchmark indicators achieved?
7. How well has the Y4P Activity delivered against its outcomes? If any, what are the areas of underperformance, why and what could have been done differently? 8. To what extent did the Y4P Activity consider cross-cutting issues of conflict sensitivity, social cohesion, and gender opportunities for women and youth regarding empowerment?
| 9. What is the quality and timeliness of Y4P Activity outputs and donor deliverables?
10. What were the value for money considerations Y4P Activity conducted?
11. How well, if at all, did the Y4P Activity learn or adapt its interventions or approaches during implementation?
12. Were the inputs appropriate towards producing the intended outputs and outcomes?
| 13. What are the overall main impacts of the Y4P Activity for its participants and other key stakeholders?
14. Were there any unforeseen positive/negative effects of the activities on the participants or communities? What are they? | 15. To what extent were the Y4P Activity interventions aligned with policies and programs of the Government and other partners operating within the same context and sector?
16. What have been the synergies between the Y4P Activity and the other CRS interventions/projects? Are there missed opportunities? How is this mitigated?
| 17 Is the Y4P Activity intervention and its impact on the participants likely to continue when USAID assistance is withdrawn? Will the Activity strategy be adapted, replicated, or scaled up?
18. What are the lessons learned and recommendations for the Y4P Activity going forward including design, implementation, exit strategy and M&E.?
|
Methodological approach
The end-line evaluation will utilize the same methods as the baseline. It will consider qualitative and quantitative approaches in the collection of data.
A survey will be done with a random sampling of 480 households in all 24 towns of the Y4P Activity communities located in Bong, Nimba, and Montserrado Counties. Enumerators will be trained to collect data under the supervision of the Consultant with support from the Program Manager, and CRS MEAL Technical Associate. The end-line evaluation will apply a mix of quantitative and qualitative methods to assess the effectiveness of strategies and implementation interventions and provide recommendations to further improve the primary areas of conflict prevention, resolution mechanisms, and social cohesion.
6 focus group discussions (FGDs) will be facilitated by the research team.
18 key informant interviews (KIIs) will be held with various key stakeholder groups including women, youth, elders, and members of civil society and community-based groups. These will also be done with one town chief per district (informal leader), and the district commissioner of each district (formal leader). These interviews will be used to (i) understand the challenges and obstacles the leaders had in resolving conflict during the project life, (ii) ways that the project assisted them in overcoming these obstacles, (iii) whether the leaders were incorporated into the startup of the project to enhance buy-in, and (iv) triangulating the information received from community members and groups.
The results of the evaluation will be presented to the Activity management team in a reflection meeting at the CRS Monrovia office with the attendance of the Donor community, relevant stakeholders, Government Officials, Local Partners, and CRS Staff. The final report will be shared by e-mail with the CRS Team. Results will be discussed and integrated into the project’s monitoring and evaluation system.
The Y4P Activity study is scheduled to be conducted between June 3rd and July 16th, 2024, encompassing the 24 targeted communities. The study will follow a well-defined timeline:
Endline Evaluation Design
The end-line evaluation will be conducted with the utmost transparency, ensuring comprehensive stakeholder engagement. This includes consultations with a broad range of relevant stakeholders, encompassing community leaders, local authorities, and religious leaders, as appropriate. To ensure the findings’ robustness, disaggregation of evaluation data will be conducted by age, gender, and, where necessary, factors such as socio-economic status and disability.
A detailed methodology will be outlined, comparing the end-line results with the established baseline data. This comparison will assess the program’s overall impact and effectiveness. The evaluation methodology will adhere to the established guidelines set forth by both USAID’s Evaluation Policy and CRS’ Monitoring, Evaluation, Accountability, and Learning (MEAL) Policies and Procedures. The evaluation will employ a mixed-methods approach, incorporating a thorough literature and desk review alongside field research. This field research will utilize a combination of quantitative and qualitative data collection methods to provide a holistic view of the program’s effects.
To ensure a cohesive understanding of the evaluation approach, data collection, analysis, and reporting processes, the consultant(s) will collaborate closely with the CRS MEAL and Programming teams. This collaborative approach will be facilitated through inception meetings, the joint development of an Evaluation Matrix, data collection tools, and an evidence assessment framework. Regular progress checks will be conducted throughout the evaluation study to maintain clear communication and alignment.
The literature and desk review will be the consultancy team’s first step to properly understand the Y4P Activity. The CRS team will provide the:
A robust data collection strategy will be employed to gather in-depth information for the evaluation. This strategy will encompass a variety of methodologies, potentially including structured interviews, focus group discussions, and key informant interviews. Additionally, data will be collected from organizations supported by the Activity, along with groups and individuals who have directly and indirectly benefited from the project. CRS staff and implementing partners will also be included in the data collection process. To gain further context and enrich the data, direct observation through visits to supported community groups will be conducted.
The consultant(s) will be responsible for proposing the most appropriate mix of these data collection approaches. This selection will be tailored to effectively address the established evaluation questions and ensure a comprehensive assessment.
The data collection process must ensure the following:
Presentation of the report and documentation of conclusions and recommendations Following data collection and analysis, a collaborative validation process will be undertaken to ensure the comprehensiveness and credibility of the evaluation’s key findings and recommendations. This process will involve the consultant(s) working closely with the Program Manager to jointly validate these aspects of the evaluation. To foster continuous improvement and knowledge sharing, a reflection meeting will be convened to discuss the lessons learned throughout the evaluation process.
The report will be structured as follows:
Evaluation roles and responsibilities:
Consultancy Main Activities
Preparation | Data Collection and Analysis | Follow up |
1. Inception meeting with the CRS team.
2. Discuss and review outcome indicators and previous baseline approaches and results.
3. Co-develop an Evaluation Matrix, data collection tools, and a common evidence assessment framework with CRS.
4. Develop a detailed work plan for the assignment, including sampling strategy and data collection plans (including one-on-one interviewee identification strategy).
5. Finalize one-on-one interviewee (to be provided by CRS). 6. Develop and finalize key Informant interviews and Focus group discussion guides and methodologies with CRS.
7. Propose an analysis framework for the assignment.
8. Develop and submit a detailed inception report detailing steps 1 to 7 above to serve as the general guidance for the conduct of the evaluation | 9. Conduct training for enumerators in collaboration with the CRS MEAL team.
10. Pre-test the survey tools and review/refine them for the main study. Conduct final field data collection and mop-ups where necessary.
11. Conduct one-on-one interviews, Key Informant interviews, and focus group discussions to collect the data.
12. Verify and validate data from the data collection activity (perform Data quality checks).
13. Analyze qualitative data and summarize key findings.
14. Analyze quantitative data using SPSS/STATA software, and PowerBI for data visualization, and summarize key findings
15. Present and discuss findings with the CRS MEAL and Program team. | 16. Submit weekly updates to the CRS project team. 17. Periodic Updates with the CRS Team.
18. Submit the final Endline report in English, including raw and clean datasets and dictionaries, Syntax used for the analysis of the survey data, and all interview and FGD transcripts. (The report should include a detailed description of findings for the Y4P Activity communities as well as state-level disaggregated results, and a comparison between the Baseline and Endline results along with in-depth information for the Y4P Activity Impact and link the results with the overall goal of the Activity. Visual data presentation should be included).
19. Design and deliver workshops to disseminate study results for the project and concerned staff using PowerPoint slides. This should also include reflections on how findings could inform future Projects.
20. Highlight measures in place to support the sustainability of the project. |
Evaluation timeline and deliverables:
Evaluation Timelines
Proposed activity | Timeline |
1. Develop and review protocol. · Review relevant literature and materials developed and used in the project. · Develop and share the preliminary protocol and tools with CRS Team. · Dialogue with CRS project management team. · Finalize, based on feedback from CRS, an inception report and tools including the tools | June 3 – 12, 2024 |
2. Conduct and implement the Final Evaluation · Orientation and training of the evaluation teams (enumerators) to collect data. · Test and modify evaluation tools before starting the evaluation. · Initiate and supervise the data collection. · Data cleaning and processing · Analyze data and share the initial findings with CRS MEAL and the program team. | June 13 – July 5, 2024 |
3. Data Validation Workshop | July 9, 2024 |
4. Submit a high-quality Report | July 15, 2024 |
Deliverable Item(s) | Description of deliverables |
Inception phase report | The inception phase report covers a review of the research instruments and finalization of methodology including the clean copy of the evaluation design matrix. |
Consolidate comments from Y4P Activity Team to inception report and conduct enumerators training | Submit endline questionnaire to ICT4D Specialist to update the Y4P CommCare platform and train enumerators in household data collection. |
Analyze and interpret data collected | The Consultant will submit the first draft of the data interpretation and suggest high-level findings of the endline evaluation. |
Cleaned Dataset, dictionary, and preliminary results | Data entry, cleaning, analysis, and visualization. |
Final evaluation survey first draft report | Development and presentation of the draft report for review by CRS Team. |
Final evaluation survey final report | Final report (preparation and submission) |
Required Qualifications
Proposals will be accepted from independent consultants/firms with verified previous experiences in conducting evaluations in the field of peacebuilding with a focus on youth programming in post-conflict or conflict-affected individuals. The consultants should possess the following combination of skills and expertise:
For individuals
For Consultancy Firm
Assigned Roles
Project Team
The Activity team will be responsible for providing the existing documents related to the project and logistics support to the consultant during the field data collection, and final evaluation presentation to CRS.
Evaluation Consultant
The role of the evaluation consultant is to work closely with the project team to develop the evaluation design; and its implementation – outlining the methodology, and key evaluation questions, identifying appropriate evaluation tools, developing the data collection instruments, carrying out data collection, data analysis and writing the draft and final evaluation report. The consultant will present the evaluation plan and findings to the project team. S/he also should adhere to USAID evaluation principles and standards for conducting project evaluation. The consultant is expected to make a formal presentation of the report to the project team.
Supervision
CRS MEAL team will be involved in all the phases of the final evaluation survey and will take a supervisory role in ensuring quality and data integrity. Specifically, the team will be responsible for recruiting the survey team (enumerators) for data collection and other supervisory responsibilities Payment Schedule
Consultancy Terms of Payment
Payment will be made to the consultant on a tranche basis. The first payment tranche is 50% of the agreed cost for the consultancy which covers field preparation, data collection and analysis. The final tranche of 50% will be paid upon the Y4P Activity (USAID) donor’s approval of the final evaluation report submitted.
Timeline
The evaluation is expected to be finalized within a period of not more than 31 working days, commencing June 3, 2024. The consultant will work with the team to assess the Lessons learned and recommendations of the Y4P Activity.
Application Submission Procedure
Interested individuals or consultancy firms are required to submit the following documentation:
The deadline for submission of both technical and financial proposals is 24th of May 2024. Proposal should be submitted to liberiaprocurement@crs.org with subject: USAID-Supported Youth for Peace (Y4P) Activity Endline Evaluation Application. Please note that application will be reviewed on a rolling basis.