Background
Global Context
Vision impairment is one of the most common preventable disabilities for children. Worldwide, more than 300 million children have a vision impairment that can be easily corrected with spectacles. Children with a vision impairment are at a major disadvantage in school. It is estimated that 80% of classroom learning during a child’s first 12 years occurs through vision, and children with a vision impairment have lower educational outcomes, such as lower rates of completion and literacy. Girls in LMICs are more impacted by vision impairment than boys, as their access to diagnosis and treatment is more limited, which adds to inequities.

Access to eyeglasses prevents children from growing into adulthood with poor vision and delivers lifelong benefits, improving education outcomes and increasing personal earnings. Despite this positive impact, few LMICs have successfully tackled this issue within their education and health systems. To help fill these gaps, NGOs across diverse contexts have introduced school eye health programs encompassing three main activities: 1) On-site vision screening for children at schools by teachers or nurses; 2) Further examination of children identified with vision problems by an eye health professional, who determines the childrens’ eyeglasses prescriptions or refers them for more advanced care; and 3) Provision of eyeglasses to children who need them. Initiatives from East and South-East Asia to Latin America and Sub-Saharan Africa have proven school eye health to be a simple, cost-effective approach for identifying children with vision problems and providing them with prescription eyeglasses.

However, these NGO-led programs have limitations. For example, they may rely on setting up their own parallel systems for training, monitoring, and other key functions, which can draw investments in health and education away from existing government systems. Further, many rely on project-based donor financing, which can lead to fragmented, smaller scale programs that shut down when donors shift priorities. In recognition of these limitations, there is growing alignment on the promise of accelerating access to eyeglasses through sustainable, nationally scaled, government-led school eye health programs among country governments and within the global development community; yet there remain critical knowledge gaps about how to achieve this in practice.

Bridging these knowledge gaps is at the center of EYElliance’s work. We launch country-level coalitions, called Evidence Labs, to architect a roadmap for integrating models of eyeglasses delivery, proven by INGOs, into government systems.

Testing a Government-Led Approach to School Eye Health in Liberia
In 2017 EYElliance established an Evidence Lab in Liberia at the request of the then president, Madame Ellen Johnson Sirleaf. She had recently collaborated with the LV Prasad Eye Institute (LVPEI) in Hyderabad, India to found the Liberia Eye Centre, the country’s first tertiary eye center, at the John F. Kennedy Memorial Medical Center in Monrovia. Following the center’s opening, EYElliance met with the President, herself a glasses wearer, and promoted the opportunity to maximize her investment in tertiary care by expanding her priorities to include the provision of eyeglasses. In the following months she advised her Ministers of Health and Education to develop a national plan to increase access to eyeglasses and called for EYElliance to convene a delegation of eye care NGOs with relevant in-country and programmatic expertise to provide technical assistance.

While the Government of Liberia had begun to increase investment in its eye health workforce, access to eye health services remained highly limited. In 2017, the entire country of 4.3 million people had only six practicing ophthalmologists and one optometrist, almost all of whom were in or near the capital, Monrovia. As a result, comprehensive facility-based eye health services at the primary, secondary, and tertiary levels were severely lacking – only five out of the country’s 15 counties had access to any eye health services. According to 2020 estimates, at least 480,000 people in Liberia have vision loss, with over 250,000 that could benefit from prescription glasses.

In developing their national plan, the Ministries of Health and Education therefore sought strategies for increasing access to eyeglasses that could rapidly and cost-effectively reach national scale while broader investments in eye health continued to ramp up. Recognizing both the wealth of evidence from NGO-led efforts and the president’s political will to endorse a national program, the Ministries set out to design and test a government-led approach to school eye health as a core component of the national plan. EYElliance has supported the Ministries to operationalize their phased national school eye health plan by: 1) convening organizations or businesses with the relevant expertise to fill capacity gaps as identified by the Ministries; 2) leveraging the existing infrastructure to minimize costs; 3) systematizing training and data collection; and 4) providing an exit for philanthropy.

Since this inception, Liberia has made great strides in establishing an enabling environment for students and teachers to receive services through its National School Eye Health program. As an initial step, the government appointed a School Eye Health Technical Working Group, convened on a regular basis by the Assistant Minister for Student Personnel Services and attended by the Director of School Health (MOE), the Director of Inclusive Education (MOE), the Program Manager of Eye Health (MoH), EYElliance, LVPEI, Sightsavers, and OneSight. With guidance from the Technical Working Group, the government designed the national program based on best practices laid out in the broadly endorsed Standard Guidelines for Comprehensive School Eye Health Programmes. Further, in 2018 the Government of Liberia integrated the National School Eye Health Program into its School Health Policy and Essential Package for School Health. Subsequent successes in program delivery led the Ministry of Education to spotlight School Eye Health as a “key result” in its 2020 Joint Education Sector Review. More recently, the Ministry of Education incorporated the National School Eye Health program into its forthcoming 2022/23-2026/27 Education Sector Plan as part of an integrated school health package.

To-date, more than 212,312 students and 5,873 teachers across 6 of Liberia’s 15 counties have received vision screening through the National School Eye Health Program, and 1,192 students and 1,687 teachers have received glasses. With support from EYElliance and the Liberia Consortium, the Ministries of Education and Health are on track to provide vision screening and eyeglasses, when needed, to approximately 85% of children enrolled in public and community schools at the primary and secondary levels by the end of 2022. Plans for securing the short-, medium-, and long-term financing to reach and sustain full national scale are under discussion as part of the Ministry of Education’s current education sector planning and budgeting process.

Evaluation Overview
Purpose and Scope of the Evaluation
The purpose of this evaluation is to assess the extent to which school eye health programming has been integrated into the Government of Liberia’s existing education and health systems at the national and county level. A secondary output will be the identification recommendations to further strengthen this integration. Additionally, it is expected that the evaluation will assess EYElliance’s model for promoting, capacitating and facilitating the Liberian government’s development and strengthening of its national school eye health program. Finally, the evaluation also aims to strengthen the evidence base for government-led school eye health programs in other countries. The evaluation will focus on the project period 2018 to 2022. The primary audiences for the evaluation are expected to include: current and prospective funders and advocates for school eye health in Liberia and globally; current and prospective school eye health stakeholders in other countries; and school eye health stakeholders in Liberia.

Evaluation Objectives and Criteria
The key evaluation objective is to examine how the government is integrating, systematizing, and institutionalizing school eye health to increase reach and sustainability. The evaluation will use the normative framework set out by the Organisation for Economic Co-Operation and Development (OECD)’s Network on Development Evaluation. The evaluation will focus on six evaluation criteria – relevance, coherence, effectiveness, efficiency, impact and sustainability. This is in line with frameworks for other recent evaluations of school health programs in Liberia.

The intervention is defined as “the integration, systematization, and institutionalization of school eye health into government systems for greater reach and sustainability.” The evaluation will be guided by the following questions pertaining to these respective criteria:
Relevance
● How does the intervention address government needs and priorities?
● How, if at all, does the intervention contribute to global goals and generate strategic significance beyond Liberia?
● What tensions and tradeoffs, if any, has the intervention encountered with regard to whose needs and priorities are met?
● Do target stakeholders, primarily government officials, view government-led school eye health programs as useful and valuable?
● Are the objectives of the intervention adequately defined, realistic and feasible? Does government leadership understand what is required to ensure integration, systematization, and institutionalization into its health and education systems?
● How has the approach adapted over time in response to any changes in external or internal context, needs, and priorities? How did these adaptations mitigate risks to a government-led approach or maximize opportunities to better meet needs and priorities?

Coherence
● How, if at all, does government leadership of school eye health programming align with the wider policy frameworks of relevant institutions?
● How, if at all, does government leadership of school eye health programming harmonize with other relevant interventions, mitigate duplication of effort, and achieve complementarity?
● How, if at all, has government leadership of school eye health programming been designed and supported to work within existing systems and structures (e.g., coordination mechanisms at the country or sector levels)?

Effectiveness
● To what extent is the government-led approach to school eye health meeting its objectives and desired results at all levels of the results chain (i.e., inputs, outputs, outcomes, impact)? (Note: focus on closely attributable results, not higher-level/broader impact)
○ What government-led systems could school eye health be integrated or embedded into? Which of these systems has it been embedded into?
○ What role did EYElliance play in ensuring integration into those systems? What was the value add of the EYElliance approach to technical assistance?
● Has government leadership for school eye health been successfully established in Liberia?
○ What factors (internal and external) influenced the level of achievement of objectives and results?
○ What mechanisms are effective for overseeing school eye health at the national and county level?

Efficiency
● What external resources were required for the intervention?
● Can the intervention’s resources (e.g., time and funding) be justified by its results, relative to the status quo (status quo defined as NGO-model implementation or doing nothing)?
● How well did the intervention utilize resources during implementation?
○ What government resources were allocated to implementation of school eye health programs?
○ What supplemental external resources were required to implement the intervention (e.g. EYElliance, LVPEI, other inputs)?
○ Were the government and external human and financial resources used as planned and appropriately and fully utilized (or were resources misallocated, budgets underspent, overspent)?
○ To what extent did the intervention achieve milestones within the intended timeframe?
○ Have intervention timelines been realistic or appropriate, and have they been reasonably adjusted in response to external factors and changes to the program?

Impact
● How, if at all, has a government-led approach to school eye health caused a significant change in access to eyeglasses for students and teachers?
● How, if at all, has the intervention generated more equitable access relative to the status quo (status quo defined as NGO-model implementation or doing nothing)?
● Is the intervention transformative – does it create enduring changes in norms and systems?
● How, if at all, is the intervention leading to intended or unintended changes, including “scalable” or “replicable” results?

Sustainability
● How, if at all, has the intervention ensured the strengthening of systems, institutions or capacities to support future development or humanitarian activity?
○ i.e. capacities, ownership/political will, budgetary commitment, policy/strategy change, legislative reform, institutional reform, governance reform, accountability, processes for public consultation
● How likely is it that any planned or current positive effects of the intervention will continue, assuming that current conditions hold? What are relevant conditions for continuation of benefits (i.e. institutional, economic/financial, environmental, political, social, and cultural stability)?

Overall
● What recommendations does the evaluator have to increase the relevance, coherence, effectiveness, efficiency, impact, or sustainability of government-led school eye health in Liberia?
● What recommendations does the evaluator have to maximize the applicability of evidence and learning from Liberia to other country contexts?

Methodology
The Evaluation will be undertaken by an independent consultant with expertise in public sector interventions and guided by an Advisory Group convened by EYElliance and the Ministry of Education. The evaluation should employ a combination of both qualitative and quantitative evaluation methods and instruments. The evaluation will require extensive review of existing documentation, as well as consultations/interviews with a sample of key partners and key local stakeholders in-country. The focus will be to triangulate information from documents and interviews by gathering objective data on key achievements and areas for improvement.
● Desk reviews: The Evaluation consultant will conduct desk reviews of relevant project documents and related documents such as routine data and monitoring reports, Standard Operating Procedures, ToRs, project progress reports, and relevant review and evaluation reports, lessons learned studies, and other analytical studies. The EYElliance Team will provide the consultant with necessary project documents and guidance on data sources. While EYElliance will aim to provide background documents, the consultant is also expected to identify and use resources via electronic and print media and solicit additional information and resources from government and partner organizations, as appropriate.
● Semi-structured interviews and focus group discussions with key stakeholders including key government counterparts and implementing partners: Drawing from the OECD framework evaluation questions, the consultant will conduct key informant and focus group discussions with stakeholders and beneficiaries, which may include county level government stakeholders, principles and teachers. All interviews should be undertaken in full confidence and anonymity. The final evaluation report should not assign specific comments to individuals.
● Field visits and on-site validation of key tangible outputs and interventions: The evaluator is expected to follow a participatory and consultative approach that ensures close engagement with the evaluation managers, implementing partners, and direct beneficiaries. At the end of visits, the Evaluator is expected to present initial data and findings for validation.
● Finalizing reports: The final phase of the evaluation will include a discussion of the findings and the draft report with the Advisory Group, the subsequent finalization of the evaluation report. The report should specifically highlight key lessons learned and good practices that could be replicated in future programs and geographies.

Evaluation Ethics:
This evaluation will be conducted in accordance with standard evaluation ethics procedures. The consultant must safeguard the rights and confidentiality of information providers, interviewees and stakeholders through measures to ensure compliance with legal and other relevant codes governing collection of data and reporting on data. The consultant must also ensure security of collected information before and after the evaluation and protocols to ensure anonymity and confidentiality of sources of information where that is expected. The information knowledge and data gathered in the evaluation process must also be solely used for the evaluation and not for other uses without the express authorization of EYElliance and partners.

Duties and Responsibilities
The Consultant will be responsible for the following: Development of a report evaluating a government-led initiative to integrate, systematize, and institutionalize school eye health to increase access to eyeglasses for students in Liberia.

Deliverables and Reports
The main deliverables of this consultancy will be:
● Inception report: The consultant will provide the evaluation design, methodology and detailed work plan to the Advisory Group.
● Draft evaluation report: The consultant will submit and present a draft evaluation report to the Advisory Group for comment and consideration.
● Final evaluation report: The consultant will submit a final report outlining the key findings, lessons learned, and recommendations, as well as an annex which is to include summaries and other background material that informed the study.

The suggested table of contents of the evaluation report is as follows:
1. Title and opening pages
2. Project and evaluation information details
3. Table of contents
4. List of acronyms and abbreviations.
5. Executive summary
6. Introduction
7. Description of the intervention
8. Evaluation scope and objectives
9. Evaluation approach and methods.
i. Evaluation approach
ii. Data sources
iii. Data-collection procedures and instruments
iv. Stakeholder participation
v. Ethical considerations
vi. Background information on evaluators
vii. Major limitations of the methodology
10. Data analysis
11. Findings
12. Conclusions
13. Recommendations
14. Lessons learned
15. Report annexes, including examples of systems integration (e.g., policies, MEL plans, SOPs, database screenshots, etc.)
16. Catalog of all tools used and data collected

Location and Travel
Monrovia, Liberia: The consultant is expected to be based in Liberia. The consultant is also expected to travel within Liberia to perform key informant interviews and focus group discussions as needed.

Period of Performance – Daily Rate
The period of performance for this Scope of Work is from January-March 2023, with the possibility of extension, depending on need. The number of work days allocated for this assignment is 30 days. Daily rate will be determined in conjunction with the responsibilities listed in this Scope of Work.

Consultant Qualifications
The assignment will require a consultant with:
● Expertise in social research methods (quantitative and qualitative research)
● A good understanding (based on previous experience) of the main approaches, methods, and tools used to evaluate public sector interventions;
● Experience of working with / for governments, international organizations or other public sector organizations;
● Evidence of applying different quantitative research techniques (e.g. statistical analysis, sample survey design, and implementation) and qualitative research techniques (e.g. interviewing, focus groups, workshops, case studies)
● 5 years minimum experience of evaluating policy, programs and / or services
● Master’s degree, preferably in public health or a related field, with strong experience in program monitoring and evaluation and technical writing on global health topics
● Excellent English language oral and written communication skills required
● Excellent organizational skills – detail oriented and accurate
● Excellent word processing skills and proficiency in Word, Excel, PowerPoint and Microsoft Office Suite required
● Comfort working in teams as well as acting independently in the implementation of specific tasks, multitasking and prioritizing, working under pressure, and meeting deadlines

Submission of Application
Please submit a resume/CV, and cover letter expressing how your knowledge, skills, and abilities are a direct match for the scope of work to taylor@eyelliance.org and copy maggie@eyelliance.org. Within the cover letter, please also include your estimated rate for completion of the 30-day contract. Place the “School Eye Health Evaluation Consultant” title in the subject line. CVs that do not match the Consultant Qualifications will not be considered. No phone calls please. Applications received after January 20, 2023 will be rejected.