Methods of adjustment for non-vaccine interventions in post-licensure vaccine studies in children in sub–Saharan Africa: a systematic review
1Institute of Infection, Veterinary and Ecological Sciences, Clinical Infection, Microbiology and Immunology Department, University of Liverpool, Liverpool, UK
2Global Health and Diarrhoea Research Group, Malawi-Liverpool-Wellcome Programme, Blantyre, Malawi
3School of Life Sciences and Allied Health Professions, Medical Laboratory Sciences Department, Kamuzu University of Health Sciences, Blantyre, Malawi
4General Medicine, Gold Cost University Hospital, Southport, Queensland, Australia
5National Institute of Health and Care Research Health Protection Research Unit in Gastrointestinal Infections
6Department of Epidemiology of Microbial Diseases and Public Health Modeling Unit, Yale School of Public Health, Yale University, USA
7Sheffield Centre for Health and Related Research, University of Sheffield, Sheffield, UK
*Corresponding author; email: latif.ndeketa@liverpool.ac.ukAbstract
Background
Post-licensure vaccine effectiveness and impact studies provide evidence on how vaccines perform under routine programme conditions in the real world. In sub-Saharan Africa (SSA), vaccine introductions frequently coincide with concurrent public health and social measures that may influence disease risk and transmission. Failure to account for these concurrent interventions may affect the interpretation of vaccine effects.
Methods
We conducted a systematic review of post-licensure vaccine effectiveness and impact studies conducted in children under five years of age in SSA. Electronic databases were searched for peer-reviewed studies published between January 2000 and December 2019. Eligible studies used observational designs to estimate vaccine effectiveness or population-level impact. Two reviewers independently screened studies, extracted data, and assessed methodological quality using Joanna Briggs Institute tools. We examined study designs, vaccines evaluated, outcomes assessed, and whether public health and social measures (PHSMs) were measured or adjusted for. A narrative synthesis was undertaken. In addition, we conducted a meta-analysis for rotavirus and pneumococcal conjugate vaccines where we explored the heterogeneity in individual-level effectiveness estimates where designs and outcomes were comparable.
Results
Sixty-four studies met the inclusion criteria, covering eight vaccine-preventable diseases. Rotavirus vaccines were most frequently evaluated, followed by pneumococcal conjugate vaccines. Case-control and ecological designs were most common, while cohort and time-series analyses were less frequently used. None of the included studies collected, reported, or adjusted for PHSMs such as nutrition, WASH, or access to healthcare. The implications of this omission varied by pathogen. Rotavirus vaccine effectiveness estimates from comparable individual-level designs were consistent across settings, with no evidence of between-study heterogeneity. Pneumococcal vaccine effectiveness estimates showed substantial heterogeneity, which appeared to reflect differences in outcome definitions, host risk profiles, and study context. Estimates for other vaccines were generally protective in direction, although the magnitude and precision varied across studies.
Conclusions
Post-licensure vaccine effectiveness and impact studies in SSA rarely account for concurrent PHSMs. The consequences of this omission are not uniform across vaccines. For some pathogens, effectiveness estimates appear robust to unmeasured contextual change, while for others they are highly sensitive to outcome choice and setting. Future evaluations should prioritise systematic measurement of key PHSMs and consider study designs that better account for time-varying context. Strengthening routine data systems to capture these factors is essential for generating interpretable evidence to inform immunisation policy.
Funding
MRC Discovery Medicine North (DiMeN) Doctoral Training Partnership (UKRI), National Institute for Health and Care Research (NIHR) Global Health Research Group on Gastrointestinal Infections and Wellcome through the core grant to the Malawi-Liverpool-Wellcome Research Programme.
Article notes
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
This systematic review was funded by the UK MRC Discovery Medicine North (DiMeN) Doctoral Training Partnership (UKRI), National Institute for Health and Care Research (NIHR) Global Health Research Group on Gastrointestinal Infections and Wellcome through the core grant to the Malawi-Liverpool-Wellcome Research Programme.
Introduction
Background of vaccine preventable diseases
Vaccination remains one of the most effective public health interventions for reducing childhood morbidity and mortality. National immunisation programmes in sub-Saharan Africa (SSA) have progressively introduced vaccines targeting major childhood pathogens including tuberculosis, diphtheria, tetanus, pertussis, poliomyelitis, pneumococcal disease, rotavirus diarrhoea, measles, and more recently typhoid and malaria(1). These vaccines have contributed to substantial declines in vaccine-preventable diseases across the region(2–5). Despite this progress, vaccine-preventable infections continue to account for a substantial burden of child mortality, with the majority of deaths occurring in low-income settings where health system constraints and high transmission intensity persist(4). The introduction of many vaccines has occurred alongside broader improvements in child health, including expanded malaria control, improved nutrition, HIV prevention programmes, and water, sanitation and hygiene (WASH) interventions. These concurrent public health and social measures (PHSMs) may influence disease burden and complicate attribution of observed reductions to vaccination alone when evaluating vaccine effectiveness or impact.
Overview of post-licensure study designs for vaccine evaluation
Pre-licensure clinical trials evaluate vaccine safety and efficacy under controlled conditions and are widely regarded as the gold standard for establishing direct protective effects(6). Individually randomised trials estimate vaccine efficacy in vaccinated individuals compared with controls, while cluster-randomised trials can also capture indirect protection by allocating communities or geographical areas to vaccination or control groups(6–8). These trials are designed primarily to support licensure and therefore typically include strict eligibility criteria and relatively defined endpoints, which may not fully reflect vaccine performance in routine use.
Post-licensure studies therefore provide essential evidence on how vaccines perform once introduced into national immunisation programmes across diverse populations and epidemiological settings. These studies can quantify direct protection in vaccinated individuals as well as broader population effects arising from reduced transmission, including indirect and overall programme impact(9,10). A range of observational designs are used for this purpose, including ecological analyses, time-series approaches, case-control studies, cohort studies and screening methods. Such designs are often necessary because withholding an approved vaccine from populations in need would raise ethical concerns. Observational evaluations nevertheless introduce additional methodological challenges, including confounding from concurrent PHSMs and broader changes in population health that occur alongside vaccine introduction.
Challenges in evaluating vaccine effectiveness
There have been observed differences in vaccine effectiveness for particular vaccines in different geographical settings, depending on differing coverage of other interventions for the same disease(22–25). There is no agreed methodological standard for the design of post-licensure vaccine effectiveness and impact studies. The modifying effects of disease related PHSMs could be important when evaluating impact of vaccination programmes. Furthermore, the epidemiology of other diseases, health interventions, health systems, surveillance efforts and societal developments are continually changing and are often difficult to quantify via the routine surveillance systems common in Sub-Saharan Africa, yet these factors may all confound impact measures. This makes it a necessity to use appropriate methods when estimating vaccine-attributable effects, particularly when studying nonspecific endpoints such as mortality and symptomatic health care utilisation. In this review, we examined the principal study designs used in post-licensure vaccine effectiveness and impact studies and assessed the extent to which these studies accounted for concurrent PHSMs.
Methods
Design
This systematic review was conducted using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. It aimed to investigate the types of study designs used in post-licensure vaccine evaluations and whether they actively considered PHSMs, and if yes, how they accounted for them in study design and/or adjusted for them in analyses. The review was registered on PROSPERO for systematic reviews (registration number: CRD42023436851)
Search strategy
A comprehensive search was conducted in electronic databases to identify relevant post-licensure vaccine studies that were published in peer-reviewed journals from the year 2000 to 2019. This was done to exclude the post-licensure studies related to COVID-19 vaccines. The electronic databases included in the search were PubMed Central, EMBASE, MEDLINE and CINAHL. In addition, to identify missing papers and non-academic literature, Google Scholar was searched and citation searching of reference list of included papers was also conducted. Review articles on vaccine impact and effectiveness studies were also used to search the references for relevant publications. The search was restricted to human studies, in English language and excluded pre-prints and review articles. The search dates were from 24/05/2023 to 27/05/2023.
To develop a comprehensive search query we used Boolean operators “AND”, “OR” and “NOT” to combine Medical Subject Headings (MeSH) terms and keywords to develop a final search string: (((“Vaccination”[MeSH Terms] OR “Immunization”[MeSH Terms] OR “immuni*”[Title/Abstract] OR “vaccin*”[Title/Abstract] OR “vaccination program*”[Title/Abstract] OR “immunization program*”[Title/Abstract] OR “immunisation program*”[Title/Abstract]) AND “Africa South of the Sahara”[MeSH Terms] AND (“Infant”[MeSH Terms] OR “child*”[Title/Abstract] OR “preschool”[Title/Abstract]) AND (“Program Evaluation”[MeSH Terms] OR “Effectiveness”[Title/Abstract] OR “Impact”[Title/Abstract])) NOT “Review”[Publication Type]) AND ((humans[Filter]) AND (english[Filter]) AND (allinfant[Filter] OR preschoolchild[Filter]) AND (2000:2019[pdat]))
Study selection and screening
Studies that were identified using the search strategy were imported into Rayyan AI, a web-based tool for managing systematic reviews(26). Rayyan AI was used to create a database of records and was also instrumental in handling the large number of records from multiple databases and de-duplicating the records.
Two blinded reviewers (LN and MV) carried out the two-stage study selection by (1) screening titles and abstracts and (2) the full text review to select articles that met the inclusion and exclusion criteria. Following the unblinding procedure, discrepancies that arose at both stages of study selection were resolved by discussion, re-examination and consensus. Figure 1 shows the outline of the review.
Inclusion and exclusion criteria
Studies were included if they were: (1) Articles published in peer reviewed journals for vaccine effectiveness and vaccine impact studies; (2) Used observational methods (e.g. case-control, cohort, time series, ecological and cluster randomized designs); (3) Published between 01 January 2000 and 31 December 2019; (4) Conducted in children living in Africa and (5) were published in English.
Studies were not included if they were: (1) Clinical trials; (2) pre-prints; (3) letters to the editor; (4) conference abstracts; (5) modelling studies that did not use observational data; (6) cost-effectiveness evaluations; (7) systematic reviews; (8) unable to access full articles; (9) serological studies; (10) vaccine coverage studies; (11) carriage studies and (12) pre – post introduction evaluations that used hospital registry data and did not report effect measures.
Data extraction
A data extraction form was developed using Microsoft Excel to systematically collect relevant data from studies including title, year of publication, journal name, author names, geographical location (country), disease of interest, vaccine of interest, other confounding PHSMs, magnitude of effect of PHSMs, adjustment for PHSMs, adjustment for other vaccines, study design, setting (community/hospital), funder, effect estimates, inclusion criteria, exclusion criteria, primary end points, sample size, study period, follow-up period, study population, data collection methods, herd immunity, follow-up procedures, measures of effect and type of funder.
Quality assessment of included studies
The Joanna Briggs Institute (JBI) quality assessment tool was used by two independent reviewers (LN and MV) on the studies that met the inclusion criteria. Any conflicts that arose were resolved through re-examination and advice from a third reviewer (DH)
Data analysis
A qualitative narrative of the reviewed papers was performed to summarise the outcomes given the heterogeneity of the study designs and other outcomes. The characteristics of studies were described in tables based on variables including, year of publication, vaccine of interest, study design, PHSMs assessed and other risk factors. For measures of vaccine effectiveness and impact we report point estimates and range of point estimates across studies. We have included meta-analyses and forest plots of effectiveness in case-control studies by vaccine type to assess between-study heterogeneity. The objective was not to generate pooled vaccine effectiveness estimates, which have been reported in other systematic reviews of specific vaccines. Instead, the analysis was intended to assess how reported effectiveness varies across comparable study designs and settings, and to consider whether differences in analytical approaches and contextual factors may contribute to this variation. Adjusted Odds Ratios and confidence intervals reported from case-control studies were included. We used random effects models with inverse variance method, using the DerSimonian-Laird between study heterogeneity estimator. Heterogeneity was measured using chi-squared (χ2) heterogeneity p-values and I2 statistics.
Results
Study selection
The search across electronic databases is presented in detail in figure 1 and yielded 64 studies included in the review. The majority were excluded because the study design and outcome did not meet the inclusion criteria.
JBI quality assessment results
The quality of the methods of included studies was moderate to high across all study designs. Using the Joanna Briggs Institute appraisal tools, none of the 64 studies were classified as poor quality by either reviewer. Most studies clearly defined their study populations, outcomes, confounders, analytical approaches, and applied appropriate statistical methods for their respective designs. However, despite generally strong internal validity within designs, none collected, reported, or adjusted for concurrent PHSMs.
Study characteristics
Our review included 64 studies evaluating post-licensure vaccine effectiveness across sub-Saharan Africa, revealing patterns in research approaches and focus areas detailed in Table 1.
Rotavirus vaccines were most frequently evaluated, accounting for 31 studies (48.4%), many conducted through the WHO-coordinated African Rotavirus Surveillance Network across multiple countries in Eastern and Southern Africa(27). Pneumococcal conjugate vaccines were assessed in nine studies (14.1%), most commonly PCV13, with fewer studies evaluating PCV7 and PCV10. Measles-containing vaccines were examined in eight studies (12.5%). Influenza and Haemophilus influenzae type b vaccines were less frequently studied, with five and four studies respectively. Meningococcal vaccines were evaluated in three studies, while polio and cholera vaccines were rarely assessed, each represented by two studies (3.1%).
The geographical distribution of studies revealed notable disparities across sub-Saharan Africa. Southern Africa accounted for nearly half of all studies (n = 30, 45.5%), followed by West Africa with 20 studies (30.3%) and East Africa with 12 studies (18.2%). In contrast, Central Africa (6.1%) and Northwest Africa (1.5%) were significantly underrepresented. In terms of publication period, there was a marked increase in study output over the last decade, with 86% (n = 57) of studies published between 2010 and 2019. Only nine studies (13.6%) were published between 2000 and 2009, reflecting both the acceleration of vaccine introductions in the region and an apparent expansion in regional research capacity.
Over half of the studies (53%) were funded by non-profit organisations such as WHO AFRO, Gavi, the Vaccine Alliance, and the Bill and Melinda Gates Foundation often in collaboration with public institutions (15%). Public sector funding alone accounted for 12%, while support from the pharmaceutical industry was limited, representing less than 5% of included studies.
Table 3 summarises the study designs used across the 64 included studies and their consideration of public health and social measures. None of the studies collected or reported data on interventions such as WASH, nutritional improvements, or access to healthcare. Case–control studies were the most common design, comprising 29 studies (45.3%), followed by ecological analyses (n = 24, 37.5%). Neither design accounted for these measures. Time-series approaches were used in seven studies (10.9%), while cohort designs were least represented (n = 4, 6.3%). Two studies combined designs, and classification was based on the primary analytical approach. Although time-series and cohort designs may offer greater scope for causal interpretation, none incorporated information on concurrent public health and social measures. No studies used the screening method or cluster-randomised designs.
Vaccine effectiveness and impact
Rotavirus vaccines reported effect estimates were consistently protective in direction, although the magnitude of both effectiveness and impact varied across study designs, outcome measures and population. Case-control and test-negative studies most often reported VE of 52% – 78% against severe or hospitalised laboratory-confirmed rotavirus(30,32,33,92–96). Ecological studies, including interrupted time-series designs, reported similar declines in diarrhoeal hospitalisations following vaccine introduction with reductions in rotavirus positivity ranging from 35% – 81%(29,45,46,49,52,54,57,97–105). Two studies found no evidence that malnutrition or HIV exposure altered rotavirus vaccine effectiveness(37,106). There was one cohort study that reported a VE of 34% against rotavirus-associated diarrhoea mortality in Malawi(40). These findings suggest that the direction of rotavirus vaccine effects ware consistent across study designs. Differences in reported effectiveness and impact may reflect variation in outcomes and epidemiological context.
Pneumococcal conjugate vaccine studies reported variable effects across outcomes and study designs. Ecological analyses generally indicated substantial reductions in invasive pneumococcal disease and meningitis following vaccine introduction, with reported effectiveness estimates often exceeding 75%(62,67,107). Individual-level case-control studies showed lower effectiveness against broader clinical pneumonia outcomes but higher protection against invasive pneumococcal disease, with estimates ranging from around 57% to 85% depending on HIV status and outcome definition(59,61,65). Time-series analyses also showed heterogeneous population-level impacts. While some settings reported moderate reductions in pneumonia hospitalisations in both HIV infected and uninfected(60), others reported smaller or non-significant changes(108). There were larger declines in meningitis and pneumonia hospitalisations were observed in Zimbabwe (109). Overall, variation in reported PCV effects appears to reflect differences in outcomes, HIV status and vaccine valency.
Other vaccines including measles, influenza, meningococcal, Hib, cholera, and polio vaccines generally showed protective effects. Measles vaccine studies consistently reported high levels of protection across study designs. Case-control evaluations from Uganda estimated vaccine effectiveness of around 74–75%(75,110). A screening method analysis conducted during an outbreak investigation in Sierra Leone reported a similar effectiveness of 74%(111). Ecological analysis from Malawi also indicated strong protection, with effectiveness estimated at 83.9% for a single dose and 90.5% for two-dose schedules. Together these findings indicate that protection from measles vaccination remains strong across different countries in SSA. Estimates were more variable in outbreak settings.
Influenza vaccine studies showed much more variability in effect estimates across seasons and settings. Case-control studies in South Africa reported effectiveness estimates ranging from −14% to 67% across five influenza seasons, with wide confidence intervals in several years(84,112,113). A study from Kenya reported more consistent protection. Vaccine effectiveness ranged from 39% to 57% depending on follow-up duration(81). One cohort analysis from Guinea-Bissau examining non-specific effects found no evidence of reduced consultation rates among vaccinated children and suggested a weaker decline compared with unvaccinated children(83). Together these findings indicate that influenza vaccine effectiveness varied substantially across seasons and contexts, with estimates ranging from negligible or uncertain protection to moderate effectiveness depending on circulating strains, follow-up period and study design.
In contrast, Hib vaccine studies consistently reported high effectiveness against invasive disease and meningitis. Case-control evaluations estimated effectiveness of around 96–99% in Uganda and Senegal(86,88). A cohort study from South Africa reported effectiveness of 83% among fully vaccinated children, with substantially lower protection in HIV-infected children(87). Surveillance data from Kenya also indicated high effectiveness of around 88%(89). Meningococcal vaccine studies reported contrasting findings by vaccine type. Ecological analysis from Senegal evaluating the polysaccharide vaccine found no significant change in meningitis incidence among children under five years(114). In contrast, ecological evaluation of the MenAfriVac conjugate vaccine in Chad reported high effectiveness of approximately 89.6%(115). These findings suggest substantially stronger population-level impact for the conjugate vaccine compared with earlier polysaccharide formulations.
Evidence for cholera vaccines was limited but consistently indicated high protection against disease. Case-control studies evaluating oral cholera vaccines reported effectiveness of 78% for the recombinant cholera toxin B-subunit killed whole-cell vaccine (Dukoral™) in Mozambique and 86.6% for the killed whole-cell vaccine Shanchol™ in Guinea. These findings suggest substantial protection across settings, although the small number of studies limits broader conclusions about variability in vaccine impact across epidemiological contexts(90,91).
Evidence for polio vaccines was limited to two case-control studies but indicated moderate to high effectiveness against paralytic disease. Estimates varied by vaccine formulation and dose number. In Nigeria, monovalent oral polio vaccine (mOPV1) showed substantially higher effectiveness (67%) against type 1 poliovirus than the trivalent formulation (16%). Trivalent vaccine effectiveness remained low (18%)against both type 1 and type 3 disease(68). A matched case-control study from Somalia reported overall effectiveness of around 70%, with higher protection observed among children receiving four or more doses. These findings highlight the importance of vaccine formulation and cumulative dosing in determining observed protection against poliovirus infection(116). As with the other vaccines, none of these studies explicitly measured or adjusted for concurrent public health and social measures, despite many evaluations being conducted in settings undergoing rapid contextual change.
Heterogeneity in effectiveness estimates: rotavirus case study
For rotavirus, 12 case-control and test-negative studies reported ratio-based effectiveness estimates against hospitalised or severe laboratory-confirmed rotavirus gastroenteritis in children under five. These studies used comparable individual-level designs and outcomes. Effect estimates were consistent across settings. In a random-effects meta-analysis, the pooled odds ratio was 0.42 (95% CI 0.35–0.51). No between-study heterogeneity was detected (I² = 0%, τ² = 0; X² p = 0.67). Several small studies reported wide confidence intervals. This uncertainty reflected limited precision within individual studies rather than differences between studies once design and outcome definitions were aligned.
Variation in post-licensure vaccine effectiveness and impact was not uniform across vaccines. It reflected differences in pathogen biology, outcome specificity, and contextual modifiers(Supplementary Table S1). In settings where public health and social measures change over time yet remain unmeasured, this distinction shapes how vaccine impact evidence should be interpreted and compared across studies.
Discussion
Summary of key findings
To the best of our knowledge, this is the first systematic review to assess the extent to which post-licensure vaccine effectiveness and impact studies conducted in children under five in sub-Saharan Africa account for PHSMs in study design and/or analysis. The key finding of this review is that none of the 64 studies across eight vaccine-preventable diseases reported or adjusted for the effects of PHSMs such as improved WASH practices, nutritional supplementation, or improved healthcare access. This represents a methodological gap in methodology for vaccine evaluations as PHSMs are known to modify vaccine effect estimates (117).
All studies included in this review used observational designs, most commonly case-control and ecological approaches. Case-control studies allow adjustment for individual-level confounders, though this relies on the assumption that cases and controls are comparable with respect to unmeasured exposures or that relevant confounders are adequately captured. This assumption may not hold for PHSMs, which are often not measured at the individual level and may vary over time and across populations. Time-series and cohort designs were less frequently used, which may reflect limited availability of pre-vaccine data, constraints in routine data systems, and the resources required to implement longitudinal studies.
The absence of adjustment for public health and social measures did not have uniform implications across vaccines. In the Results, rotavirus effectiveness estimates from comparable individual-level designs and outcomes were stable across settings, despite the absence of measured PHSMs. Pneumococcal conjugate vaccine estimates varied across outcomes and contexts, even within similar case-control designs. Influenza estimates showed pronounced seasonal and strain-related variation, while Haemophilus influenzae type b and meningococcal conjugate vaccines were associated with large and sustained reductions in disease in high-coverage settings. These patterns suggest that unmeasured PHSMs may influence vaccine effects in ways that differ by pathogen, rather than acting as a uniform source of variability.
Recommendations for future studies
Future post-licensure vaccine effectiveness and impact studies in sub-Saharan Africa should prioritise improved measurement of PHSMs that plausibly modify vaccine effects and change over short time scales. Case-control studies should collect individual-level data on nutrition, WASH exposure, malaria prevention, and other contextually relevant factors that are known to influence susceptibility or immune response. These variables should be incorporated for matching in design or in analytical models to allow estimation of effect modification rather than treated solely as sources of confounding.
Matching or adjustment for broader factors such on geography, seasonality may provide partial control for contextual variation and can be considered as complementary strategies. However, these approaches do not replace direct measurement of rapidly evolving PHSMs. In ecological and time-series evaluations, strengthening routine data systems such as District Health Information Systems is crucial for capturing population-level indicators of nutrition, WASH and other key interventions, enabling better interpretation of vaccine impact alongside changes risk factors. As climate variability affects disease seasonality and risk, improving the capture of environmental and social indicators will become more important.
Cohort studies should prioritise the collection of time-varying PHSMs data and adjust for PHSMs accordingly. For interrupted time-series analyses, linkage aggregated health data with routine sources such as WASH databases or HIV programme records can improve adjustment for concurrent interventions and reduce residual confounding. Ministries of Health should be supported to strengthen surveillance platforms incorporating indicators of PHSMs into routine health information systems to improve monitoring and vaccine impact assessment. Where PHSMs change rapidly and are not captured at the individual level, alternative evaluation designs may be required.
One such approach is cluster randomised evaluation designs as used in the RTS,S/AS01 malaria vaccine pilot implementation and evaluation (MVIE) in Malawi, Ghana and Kenya (122). This allowed unmeasured contextual factors such as health system strengthening, care-seeking behaviour, and concurrent malaria interventions such as bed net usage and indoor residual spraying to be balanced across study arms by design. While this design doesn’t eliminate effect modification at the individual level, it allows estimation of population-average treatment effects that are less sensitive to confounding by time-varying and unmeasured PHSMs. While cluster randomised post-licensure evaluations are not routinely feasible, the RTS,S experience illustrates that such designs can be justified where safety, effectiveness, and programme impact need to be assessed under real-world conditions and where conventional observational designs are likely to yield unstable estimates.
Implications for policy and practice
Design limitations can lead to underestimation or overestimation of vaccine effect estimates. This has implications for immunisation programmes in low-resource settings, where competing priorities for limited resources may lead to missed opportunities to identify and prioritise synergistic interventions that could enhance the overall impact of vaccination. For example, governments could combine immunisation campaigns with nutrition interventions or distribution of insecticide treated bed nets. For diarrhoeal disease such as cholera, reactive vaccination campaigns should be implemented alongside WASH interventions. It is crucial to collect and adjust for data on PHSMs implemented alongside vaccination programs during evaluations.
Neglecting PHSMs may lead health authorities in LMIC settings to place excessive emphasis on vaccination alone, sidelining complementary interventions such as nutrition programmes, WASH improvements, and maternal health education. At the same time, underestimation of vaccine effects may result in deprioritising effective vaccines, risking robust coverage and outbreak control. Vaccination remains among the most cost-effective public health interventions across all settings(123). Delaying or abandoning vaccine introduction based on robust effect estimates may increase costs for both households and health systems while leaving populations vulnerable to preventable disease.
Underestimated vaccine impact may undermine public and political confidence in vaccines, especially in the face of rising vaccine hesitancy as experienced with COVID-19 vaccines(124). Adjusting for PHSMs for post-licensure studies is also an opportunity for LMICs to expand national data platforms like DHIS to routinely capture PHSMs indicators. Health facility registers and survey questionnaires could be updated to include nutrition, WASH, and other co-interventions to improve evidence quality and support integrated programme planning.
Strengths and limitations
The review employed a comprehensive and reproducible search strategy across multiple databases including PubMed Central, EMBASE, MEDLINE, and CINAHL and adhered to PRISMA guidelines. We included a wide range of observational study designs, funding sources and vaccines which allowed for a comprehensive appraisal of the methodological approaches employed in post-licensure vaccine evaluations. Furthermore, our dual-reviewer approach to screening, extraction, and quality assessment enhanced the rigour and reliability of the review process.
However, our systematic review had several limitations. The review was restricted to English-language publications, which may have led to the exclusion of relevant studies conducted in francophone and lusophone countries within the region which also have strong immunisation programmes. The review was also limited to countries in sub-Saharan Africa, which excludes evidence from other regions and may limit the generalisability of the findings beyond this context. We relied on published data where analyses were constrained by what was reported in the articles. It is possible that some researchers measured PHSMs but simply did not report them. Finally, we limited our review to 2019 which excluded the evaluation of COVID-19 vaccines.
This scope was chosen to focus on post-licensure evaluation under routine immunisation conditions. COVID-19 vaccine studies were conducted in a markedly different context, with rapidly changing and large-scale PHSMs introduced alongside vaccine rollout. These features create strong co-interventions and secular shifts that are not comparable with the settings in which most established childhood vaccines are evaluated. The rapid expansion of COVID-19–related publications would also have dominated the evidence base and shifted our review away from its primary objectives. Despite these limitations, our review highlights a critical methodological gap in post-licensure vaccine studies and underscores the urgent need for future evaluations to systematically account for PHSMs in both design and analysis.
Conclusion
This systematic review highlights a critical and under-addressed gap in post-licensure vaccine effectiveness and impact studies conducted in children under five years in sub-Saharan Africa which is the lack of adjustment for PHSMs such as improved nutrition, WASH, malaria control, and HIV-related services. None of the 64 studies included in this review collected, reported, or adjusted for PHSMs in their analysis despite the known confounding effects of these interventions on disease burden and health outcomes. With increasing number of vaccines and broader demands on health budgets it is necessary to provide the most accurate assessments of vaccine contributions to public health in rapidly changing settings.
It is critical to improve the design and reporting of post-licensure vaccine studies in LMIC settings. This includes routine collection of PHSMs-related data, employment of stepped-wedge cluster randomised design and incorporation of such variables into multivariate regression models. Additionally, national health information systems should be strengthened to facilitate the routine capture of relevant PHSMs by researchers. Integrating these improvements into surveillance and evaluation frameworks will improve accuracy and provide context-specific evidence to guide immunisation policies and maximise public health gains from vaccination programmes.
Supporting information
Data Availability
All data produced in the present study are available upon reasonable request to the authors