In our second piece of this exciting feature on Riders For Health, Matthew Hann dives into their new health program, looking into how they benchmark their data.
In September, Markets for Good published the first in a series of articles about Riders for Health’s new project in Lesotho. Having worked in the Southern African country since 2009, this new project sees us partner with UNICEF, the Jersey Overseas Aid Commission and the Lesotho Ministry of Health. Our goal is to mobilise teams of health workers in Berea and Mohale’s Hoek whose role will be to carry out immunization in rural, near impossible to reach, communities.
Before the project began
Introducing a new project, wherever it is in the world is expensive. In countries where resources are already scarce and where funding for health care is limited, it is even more important that we are able to say whether money is being invested in the right way.
Is what we are doing working? And if not, why not? Is there anything that we, or our partners can change to improve it? Or are there unintended consequences that will need to be corrected as a result?
To be able to get a true sense of how a programme impacts over time, it is important to have as good a sense as possible of the picture before the work started. We call this a baseline study.
For this project the baseline study was completed with the help of our partners, UNICEF and the Ministry of Health. We wanted to find out what health workers were currently able to do, how far they were able to travel and what communities were out of their reach.
The baseline information was obtained from health centres and clinics, and the nurses in charge. They comprised of ‘Health Indicators’ showing target population and data on resources including workforce from facilities, services provided and information on transport used for outreach.
The results of the baseline
Lesotho is still having challenges in attaining the target of 90% immunization coverage. Mohale’s Hoek and Berea are having greater challenges than most districts. Data collected for the baseline study showed that coverage for measles vaccination in Mohale’s Hoek was 58.5%, while in Berea it was 69.4%. Both districts were below the nationwide average of 72.2%.
The baseline showed that in Berea:
- only two facilities reported to have vehicles in good condition and punctual. For the remainder of the facilities, the condition of vehicles used for outreach was unsatisfactory.
- health facilities indicated that between one and three outreach visits were carried out in recent months.
In Mohale’s Hoek the baseline showed that:
- five facilities reported to have vehicles in good condition and punctual while four indicated their vehicles are in fair condition. For the remaining six, the condition of vehicles was reported to be unsatisfactory.
- Average time spent per patient or client was said to be about 15 minutes.
But as well as showing the state of immunization in Berea and Mohale’s Hoek, this baseline study also highlighted that there were a lot of gaps and missing immunisation information in both districts.
Some of the missing information included:
- Population estimates were found in less than half of facilities in Berea, even though providers of information in health facilities recognized its importance.
- In Bearea, there was no unified standard tool in facilities for collecting data on immunisations, since routine EPI data and surveillance are not part of the Ministry of Health HMIS in Health Planning & Statistics M&E.
- Only eleven health facilities in Mohale’s Hoek had population estimates of their underserved communities.
The report found that the availability of immunisation data was not ideal. It has shown us that to be able to assess the project properly, we need to:
- Design and standardise M&E tools for use at health centres and clinics.
- Improve the outreach schedule visits forms by making it possible to capture other valuable information around health outreach activities.
- Improve and encourage the full and effective use of data at points of collection where it is generated.
Overcoming the challenges
The results that we have to work from are not perfect, but in reality it is impossible to create a perfect baseline study. You are often working from data from a lot of different sources, in a range of formats, all collecting slightly different things.
What is more, in facilities where resources and time are limited and where IT facilities may be non-existent, records are usually incomplete or require hours of data entry.
But the baseline in Lesotho has been easier than it could have been. Riders have been working in Lesotho for several years and we have an office and an M&E infrastructure. Where we have stated working in a country from scratch, and have no existing infrastrure before the project launch, it has often been more challenging. Where there has been only limited lead time between the project being approved to it starting there is little time for a baseline study before the project is underway.
It is possible to introduce systems of M&E once a project is underway, it’s something we have done, and it is certainly better than having no system of analysing data, but you will never be able to compare the areas you are working in before you arrived.
“In reality it is impossible to create a perfect baseline study”
What happens now
The health workers have now been trained to ride safely and look after their bikes, and they are now delivering services out in the community.
They are also collecting data and it is being delivered to our team. Each health worker fills out a daily log sheet designed to record distances travelled, activities completed and so on. Every month, when a Riders technician visits to service their motorcycle, these log sheets are collected and taken back to our office in Maseru where our monitoring and evaluation officer Charmaine will enter and analyse the data.
In the coming months we will start by outlining some of the early results of our data collection as it comes through and, thanks to our baseline study we will be able to begin assessing whether the programme is having the impact we expect.
Many thanks to Matthew Hann for building on the journey of Riders For Health’s new health program, and the challenges posed by conducted a thorough baseline analysis. We look forward to hearing how the project progresses, and how Riders deal with missing data.
To stay up to date with the latest Markets For Good articles and news, sign up to our newsletter here. Make sure that you are also following us on Twitter.