Lot Quality Assurance Sampling (LQAS)

In 2024, the Mali Country Learning Hub—Centre d’Apprentissage pour l’Équité en Vaccination (CAPEV)—piloted two interventions to improve immunization delivery across four districts in Mali: Coaching de Performance (C2P) and MEDEXIS. As part of this pilot, an LQAS survey was conducted to assess baseline vaccination coverage and identify operational barriers.

Sampling Design

The survey was designed to measure performance at a highly localized level. The geographic units were defined as follows:

• Catchment Areas (CAs): These were defined as the official health facility catchment areas (aires de santé). Two CAs per district were selected based on having a large target population (at least 1,000 children under 23 months) and low Penta3 coverage.
• Supervision Areas (SAs) / Lots: Instead of treating an entire health facility catchment as one area, the SAs were defined as smaller subsections within each CA. These subdivisions directly aligned with the specific geographic zones assigned to the Community Health Workers (agents de santé communautaire). Mapping lots to these micro-zones allowed the project team to pinpoint pockets of low coverage within a single facility’s reach.

Four CAs were divided into 5 lots, while two CAs contained 4 lots due to geographic constraints. Interviews were split evenly between two target age cohorts: children aged 18 weeks–11 months and children aged 12–23 months. For CAs with five lots, the LQAS sample size was n = 19 per SA per age cohort. For CAs with four lots, the team increased the lot sample to n=24 to achieve a total N=96 (nearly the same as n=95 for the CAs with five lots).

Interview sites within each SA were selected using Probability Proportional to Estimated Size (PPES) from updated settlement lists. Households were selected using segmentation sampling and a random-start method. Security challenges in the Yorosso district ultimately required the removal of its CAs from the final sample. Data collection successfully proceeded across the remaining 6 CAs (comprising 28 total SAs), yielding a final sample of 1,144 interviews.

Indicators

The LQAS survey in Mali measured the following:

• Vaccination Coverage: Coverage for BCG, Penta1, Penta3, Measles (VAR1, VAR2), and other routine antigens.
• Zero-Dose (ZD) Prevalence: The proportion of children aged 12–23 months who had not received Penta1.
• Dropout Rates: Penta/DTP dropout between early and later doses (e.g., Penta1 to Penta3).
• Timeliness: On-time delivery of early infant vaccinations.
• Barriers to Service: Reasons for non-vaccination, including perceived stock-outs, physical access, and financial costs.
• Behavioral Determinants: Sociodemographic factors, caregiver perceptions, trust in health services, and female decision-making autonomy.

Data Use

Subdividing a single health facility’s catchment area into multiple community health worker lots allowed local teams to identify micro-level performance gaps and inequities. These localized disparities would have been otherwise masked if multiple facilities had been grouped into a single catchment area. The classification of each lot against LQAS decision rules helped managers prioritize targeted interventions, such as mobile outreach or adjusted service hours, for specific underperforming subsections of the facility’s area. This approach is particularly useful where catchment populations are large, geographically diverse or face heterogenous barriers to vaccination.

When designing an LQAS survey, programs may establish coverage thresholds and corresponding decision rules for each antigen or indicator of interest. Thresholds do not need to be the same for all antigens. Indicators with higher baseline coverage—often vaccines given earlier in the immunization schedule, such as BCG—can be assessed against higher thresholds. In contrast, indicators with lower baseline coverage—often vaccines administered later in the vaccination schedule, where dropout is more likely, such DTP3 or MCV1—may be assessed against lower thresholds. Thresholds should be informed by available baseline coverage estimates from administrative data or population surveys (e.g., DHS or MICS). When LQAS is used repeatedly in the same lots to monitor program performance, results from the initial round can also serve as a useful baseline for setting upper and lower thresholds in subsequent rounds.

Steps:

• Identify antigens or indicators of interest (e.g., BCG, Penta/DTP1, Penta/DTP3, MCV1).
• Review recent coverage estimates to understand baseline levels, including past LQAS data if available.
• Set upper coverage thresholds that are both realistic and ambitious. Upper threshold = the current coverage estimate or program target for each antigen; lower threshold = upper threshold - 30%.
• Select a decision rule for each antigen based on the LQAS Decision Rule Table for your sample size (most common is n = 19) where the combined alpha and beta errors are <=10%.

Table 1 shows how different thresholds and decision rules can be used for three childhood vaccination indicators across five SAs/lots using hypothetical data. In this case, the SAs were defined as the wards in one selected district (LGA) and the sample size in each ward was 19 children aged 12–23 months. The selection of upper and lower thresholds was based on available data from previous MICS surveys that showed lower existing coverage of Penta 1 and Penta 3 (70% and 60%, respectively) at the state level.

Table 1 shows how different thresholds and decision rules can be used for three childhood vaccination indicators across five SAs/lots using hypothetical data. In this case, the SAs were defined as the wards in one selected district (LGA) and the sample size in each ward was 19 children aged 12–23 months. The selection of upper and lower thresholds was based on available data from previous MICS surveys that showed lower existing coverage of Penta 1 and Penta 3 (70% and 60%, respectively) at the state level.

Interpretation:

– Lots that FAIL: SAs where coverage is below the stated lower threshold. SA3 in particular performed poorly across all three antigens; a program would likely focus resources in this SA to strengthen services and/or demand. Further investigation is needed to determine the root causes of the poor performance.

– Lots that PASS: SAs where coverage is at or above the stated upper threshold. SA2 and SA5 performed well across all three antigens and may provide lessons learned for SAs that have weaker vaccination performance.

Table 1. Illustrative LQAS Classification of Vaccination Coverage Using Decision Rules and Thresholds

Managers need to understand how to select LQAS thresholds and decision rules for the BeSD indicators. These indicators have been incorporated in the LQAS survey instruments used by the Nigeria and Mali CLHs for the purpose of classifying the drivers of vaccination at the subnational level. While BeSD data are mainly analyzed at the aggregate CA level, we describe how to classify BeSD data at SA level here.

To set thresholds and decision rules for BeSD indicators, you must transition from measuring coverage (a binary yes/no indicator) to measuring sentiment or experience (e.g., “was the parent treated with respect?”). Because BeSD indicators such as intent, confidence, and practical issues often precede the actual vaccination event, the thresholds you choose should reflect your programmatic “tolerance” for these barriers.

Practical Example Using the Respectful Care Indicator

In the Mali and Nigeria CLHs, a critical BeSD driver is the quality of immunization services.

Why BeSD Thresholds Differ from Vaccination Antigens

Unlike immunization coverage, where 80% is a commonly-used benchmark, BeSD thresholds are often context-specific:

• For “Intent to Vaccinate”: You might set a very high pU (90%) because intent should always be high.
• For “Practical Issues” (e.g., distance): You might set a more modest pU (70%) if the geography is known to be difficult, focusing only on identifying the absolute worst performing areas.

Note: By choosing n = 19 and the relevant threshold levels, you are ensuring that your alpha error (failing an acceptable lot) and beta error (passing an unacceptable lot) remain low, usually below 10%.