Interim Evaluation of Accelerating Access to Basic Vision Screening and Reading Glasses via Government-Led Community Health Worker Programs

CONSULTANT SCOPE OF WORK

August 2023

Background

Scope of the Problem and Potential Impact

A simple 1 USD pair of non-prescription reading glasses can transform the lives and livelihoods of those suffering from near-vision loss. Correcting near-vision loss with reading glasses not only improves personal quality of life but also generates far-reaching economic benefits. Research has demonstrated that when workers receive the reading glasses they need, they experience an immediate increase in productivity of 21.7%. In workers over the age of 50, productivity increases by up to 31.6%. Yet more than 800 million people worldwide lack access to reading glasses.

Adapting Proven Models for Government Delivery

Achieving equitable access to reading glasses is a solvable problem. Community-based models for basic eye health screening and provision of reading glasses have proven effective across diverse contexts. To-date, over 35,000 community-level workers have been trained in Bangladesh, China, Kenya, Rwanda, Uganda, and Pakistan, creating a new eye health workforce to reach those previously without access. However, these models have operated almost exclusively outside of government health systems and have consequently struggled to reach and sustain scale. To address this challenge, EYElliance’s strategy leverages existing, government-led community health worker programs to bring basic vision screening and reading glasses to the nearly one billion people who need them.

Demonstrated Success: Liberia’s National Community Health Assistant Program

To test this strategy, EYElliance partnered Liberia’s Ministry of Health to integrate these life-changing services into its National Community Health Assistant (NCHA) Program, which serves the 1.2 million Liberians living more than 5km from the nearest health facility. To achieve this integration, the Ministry of Health is embedding community eye health components across the program’s existing systems, such as training, supervision, reporting, and supply chain—beginning with a pilot in Margibi county. Within just a few months after training, Community Health Assistants (CHAs) had already screened and provided reading glasses to thousands of community members with near vision loss. Given the NCHA program’s traditional focus on reproductive, maternal, and child health, the Ministry’s introduction of eye health services has expanded services to all adults. Further, the Ministry has reported that this expansion has enhanced CHAs’ motivation and augmented community perceptions of the NCHA Program as a whole. Building on these findings, the Ministry has now formally embedded basic vision screening and provision of reading glasses into the recent revision of its community health policy and training curriculum.

Evaluation Overview

Purpose and Scope of the Evaluation

The purpose of this evaluation is to assess the integration of basic vision services into Liberia’s National Community Health Program (NCHP) at the national and county level and generate evidence and recommendations, both to inform scale-up within Liberia and to facilitate global advocacy, financing, and integration of basic vision services into CHW programs in other countries. The evaluation will focus on the project period 2020 to 2023. The primary audiences for the evaluation are expected to include: current and prospective community eye health funders, advocates, government decision-makers, managers, and other stakeholders in Liberia and globally.

Evaluation Objectives and Criteria

The overarching evaluation objective is to examine the integration and impact of basic vision screening into CHW programs, drawing from Liberia’s experience within the NCHA program.

The Consultant is expected to propose evaluation approaches for the objectives below based on desk review and stakeholder consultation. The evaluation approach for each objective will be reviewed and approved by an Advisory Group convened by EYElliance and the Ministry of Health. It is expected that the Consultant will draw from, and adapt as needed, the Organisation for Economic Co-Operation and Development (OECD)’s Network on Development Evaluation, systems integration, systems change, or other evidence-based approaches as relevant.

Objective 1: Inform Strategies for Integrating, Scaling, and Sustaining Basic Vision Services within CHW Programs in Liberia and Other Country Contexts

Objective 1 of the evaluation is to map the current depth and breadth of Liberia’s integration and institutionalization of basic vision services into the NCHP at the national and sub-national level and to generate recommendations to facilitate and strengthen full integration, scale-up, and sustainability as Liberia moves from a single county pilot to national scale. The resulting mapping and recommendations are also intended to serve as a resource for introducing and maintaining basic vision services within CHW programs in other countries.

The primary audience for Objective 1 includes: country governments who are pursuing or considering the integration and scale-up of basic vision services via CHW programs (both within and beyond Liberia), as well as relevant technical assistance and implementing partners who are accompanying governments in these efforts.

Demonstrative evaluation questions for Objective 1 include:

Relevance

• What is required for the government to ensure integration, systematization, and institutionalization of basic vision services into the NCHP and the broader health system?
• How does the intervention address government needs and priorities?
• How, if at all, does the intervention contribute to global goals and generate strategic significance beyond Liberia?
• What tensions and tradeoffs, if any, has the intervention encountered with regard to whose needs and priorities are met?
• Do target stakeholders, primarily government officials, view government-led community eye health programs within the NCHP as useful and valuable?

Coherence

• How, if at all, does the provision of basic vision services within the NCHP align with the wider policy frameworks of relevant institutions?
• How, if at all, does the provision of basic vision services within the NCHP harmonize with other relevant interventions, mitigate duplication of effort, and achieve complementarity?
• How, if at all, has the provision of basic vision services within the NCHP been designed and supported to work within existing systems and structures?
  • existing NCHP systems such as coordination, management, training, supervision, monitoring, supply chain, and health information

Sustainability

• How, if at all, has the intervention ensured the strengthening of systems, institutions, or capacities to support the continuation of the intervention?
  • e. capacities, ownership/political will, budgetary commitment, policy/strategy change, legislative reform, institutional reform, governance reform, accountability, processes for public consultation
• How likely is it that any planned or current positive effects of the intervention will continue, assuming that current conditions hold? What are relevant conditions for continuation of benefits (i.e. institutional, economic/financial, environmental, political, social, and cultural stability)?

Objective 2: Assess the Outcomes of Integrating Basic Vision Services within Liberia’s National Community Health Program

Objective 2 of the evaluation is to assess the outcomes of integrating basic vision services into government-led CHW programs by drawing on insights from Liberia. The intention of this exercise is to position advocates to make the case for integration of basic vision services and respond to common stakeholder considerations. As such, it will be critical for the Consultant to identify what impact or other data are necessary to establish and maintain buy-in both within and beyond Liberia and, as feasible within the scope of the evaluation, to collect and package this data for use by relevant stakeholders.

Demonstrative evaluation questions for Objective 2 include:

Impact

• How, if at all, has a government-led approach to delivering basic vision services within the NCHP caused a significant change in access to reading glasses for community members?
• How, if at all, has the intervention generated more equitable access to reading glasses relative to the status quo (status quo defined as NGO-model implementation or doing nothing)?
• Is the intervention transformative – does it create enduring changes in norms and systems?
• How has the intervention changed community perception of Community Health Workers?
• How has the intervention changed Community Health Workers’ perception of their role in the community?
• How, if at all, is the intervention leading to intended or unintended changes, including “scalable” or “replicable” results?

The primary audience for Objective 2 includes: current and prospective community eye health funders, advocates, government decision-makers, managers, and other stakeholders in Liberia and globally.

Methodology

The Evaluation will be undertaken by an independent consultant with expertise in public sector interventions and guided by the Advisory Group. While the methodology for each objective will vary to some degree, this section outlines cross-cutting components that will be deployed as relevant across each evaluation objective.

The evaluation should employ a combination of both qualitative and quantitative evaluation methods and instruments. The evaluation will require extensive review of existing documentation, as well as consultations/interviews with a sample of key partners and key local stakeholders in-country. The focus will be to triangulate information from documents and interviews by gathering objective data on key achievements and areas for improvement.

• Desk reviews: The Evaluation consultant will conduct desk reviews of relevant materials such as project documents and related documents such as existing tools and studies for the NCHP, routine data and monitoring reports, Standard Operating Procedures, ToRs, project progress reports, and relevant review and evaluation reports, lessons learned studies, and other analytical studies. The EYElliance Team will provide the consultant with necessary project documents and guidance on data sources. While EYElliance will aim to provide background documents, the consultant is also expected to identify and use resources via electronic and print media and solicit additional information and resources from government and partner organizations, as appropriate.
• Semi-structured interviews and focus group discussions with EYElliance staff and key stakeholders, including key government counterparts and implementing partners: Drawing from the approved inception report (see below), the consultant will conduct key informant and focus group discussions with stakeholders and beneficiaries which may include county level government stakeholders, community health workers, and community members. All interviews should be undertaken in full confidence and anonymity. The final evaluation report should not assign specific comments to individuals.
• Field visits and on-site validation of key tangible outputs and interventions: The evaluator is expected to follow a participatory and consultative approach that ensures close engagement with the evaluation managers, implementing partners, and direct beneficiaries. At the end of visits, the Evaluator is expected to present initial data and findings for validation.
• Finalizing reports: The final phase of the evaluation will include a discussion of the findings and the draft report with the Advisory Group, the subsequent finalization of the evaluation report. The report should specifically highlight key lessons learned and good practices that could be replicated in future programs and geographies.

Evaluation Ethics

This evaluation will be conducted in accordance with standard evaluation ethics procedures. The consultant must safeguard the rights and confidentiality of information providers, interviewees and stakeholders through measures to ensure compliance with legal and other relevant codes governing collection of data and reporting on data. The consultant must also ensure security of collected information before and after the evaluation and protocols to ensure anonymity and confidentiality of sources of information where that is expected. The information knowledge and data gathered in the evaluation process must also be solely used for the evaluation and not for other uses without the express authorization of EYElliance and partners.

Duties and Responsibilities

Deliverables and Reports

The main deliverables of this consultancy will be:

• Inception report: The Consultant will provide the evaluation design, methodology and detailed work plan to the Advisory Group.
• Interim drafts: The Consultant will submit and present drafts of each of the final deliverables listed below to the Advisory Group for comment and consideration.
• Final deliverables: EYElliance and the Consultant will collaborate throughout the evaluation process to assess and determine the most desirable format for final deliverables. The Consultant will be responsible for agreed-upon final deliverables as well as annexes that include summaries and other background material that informed the evaluation. Demonstrative deliverables are listed below under each objective:
  • Objective 1:
    • Report outlining the key findings and lessons learned from Liberia’s integration and institutionalization of basic vision services to-date, as well as recommendations to facilitate and strengthen full integration, scale-up, and sustainability
  • Objective 2:
    • Report outlining the key findings on the value-add of integrating basic vision services into Liberia’s NCHA program via the Margibi pilot, as well as recommendations for making the case for integration of basic vision services.

Location and Travel

Monrovia, Liberia. The consultant is expected to travel within Liberia to perform key informant interviews and focus group discussions as needed.

Period of Performance – Daily Rate

Due to the pending elections in October 2023, the proposed period of performance for this Scope of Work is from August 2023 – November 2023, with the possibility of extension, depending on need. The proposed number of work days for this assignment is up to 50 days. Daily rate will be determined in conjunction with the responsibilities listed in this Scope of Work.

Consultant Qualifications

The assignment will require a consultant with:

• Expertise in social research methods (quantitative and qualitative research)
• A good understanding (based on previous experience) of the main approaches, methods, and tools used to evaluate public sector interventions;
• Experience of working with / for governments, international organizations or other public sector organizations;
• Evidence of applying different quantitative research techniques (e.g. statistical analysis, sample survey design, and implementation) and qualitative research techniques (e.g. interviewing, focus groups, workshops, case studies)
• 5 years minimum experience of evaluating policy, programs and / or services
• Master’s degree, preferably in public health or a related field, with strong experience in program monitoring and evaluation and technical writing on global health topics
• Excellent English language oral and written communication skills required
• Excellent organizational skills – detail oriented and accurate
• Excellent word processing skills and proficiency in Word, Excel, PowerPoint and Microsoft Office Suite required
• Comfort working in teams as well as acting independently in the implementation of specific tasks, multitasking and prioritizing, working under pressure, and meeting deadlines

Submission of Application

Please submit a resume/CV alongside a proposal of how you would fulfill the scope of work, including relevant past experience, proposed work plan, and budget. All communications should be directed to abigail@eyelliance.org. Place the “Community Eye Health Evaluation Consultant” in the subject line. Resumes/CVs and proposals that do not match the Consultant Qualifications and Scope of Work will not be considered. No phone calls please. Applications will be evaluated on a rolling basis. Applications received after August 31, 2023 will be rejected.