We examined the fidelity of the implementation process and coverage of DATs and associated differentiated support under pragmatic trial conditions among adults with drug-sensitive TB in five countries. Results differ by country and provide insights into how the implementation of digital support mechanisms can be optimized in the future. Key findings indicate firstly that phone ownership was high overall. Secondly, engagement with DATs was high among participants, particularly in the pillbox arm, with digitally recorded doses ranging from 82% to 91%, with strong concurrence from the qualitative interviews. Thirdly, there was considerable variation in both method and frequency of HCPs accessing the digital adherence platform, with more than 50% of logins to the platform being from mobile phones rather than tablets or desktops in Ukraine and the Philippines. Additionally, the percentage of manual doses added to the adherence platform more than 7 days after the dose-day varied considerably by country, with the Philippines (61%), more than double Ethiopia (30%), in the pillbox arm.

The findings confirm the pragmatic nature of the trials with each country adapting their interventions to their local context. Moreover, this study provides a rich diversity of context for the implementation of DATs in terms of geographical, cultural, socio-economic, and demographic factors. Key findings indicate that integration of digital systems into daily workflow is complex, and the ability of health systems to adopt the new technology varies per country and facility. Health system and HCPs capacity has an impact on the ability to utilize real-time data. Reviewing adherence data daily may not be possible for all HCPs or necessary for all PwTB, and value may still be derived from less regular assessment of adherence and taking action where doses are missed, or prioritising adherence monitoring of certain population. DATs are a highly acceptable form of treatment support for PwTB and for the HCPs who can integrate digital systems with their regular tasks.

Access to a mobile phone was a requirement for using the medication label and the intervention can be implemented with a shared phone, although unintended disclosure of TB status is less likely on a personal phone. In an earlier study conducted in Tanzania, it was found that those with the highest engagement with 99DOTS were those who owned a mobile phone that was not shared (23). Access to mobile technology is key to the implementation of digital health interventions (24). Timely delivery of differentiated care is facilitated by a phone that is not shared. Automated SMS reminders when a digital dose was not received by the platform before a certain time each day was common but with variation between countries and more markedly DAT type. Qualitative findings on reasons to miss a dose, therefore, prompting a reminder message was aligned with other studies, where, the ability to read, technology fatigue and unstable network were cited as barriers to digital recording of a dose (25), (26), (27), (28).

Facility staff received formal training on the implementation of DATs and use of the adherence platform, varying from two HCPs per facility in Tanzania to seven per rayon in Ukraine. However, the cadre of staff trained varied. In Ukraine TB doctors would enroll PwTB on DATs, while in the Philippines, due to redeployment to COVID-19 services, additional staff from other disciplines received on the job training from TB focal persons enabling them to enroll and monitor PwTB. The lack of more comprehensive training of the staff providing secondary support in the Philippines may have impacted their ability to implement the intervention as planned. In addition, there were likely varying skill levels in the use of digital technology and in building supportive relationships with DAT users. In South Africa, research interns assisted TB focal nurses with implementation.

HCP engagement with the adherence platform was measured by several indicators. A higher percentage of weekday days with at least one visit to the platform per facility, indicates use of the platform was integrated into routine workflow, as seen in Ukraine at 76%. It is also possible that with more staff trained in Ukraine, there was a collective rather than individual action to make checking the platform routine practice, seen as key to professional behaviour change in complex healthcare settings (29). Daily reviewing of DAT data was indicated by 90% of HCPs in a meta-analysis of six studies (23), although this was a small sample self-reporting rather than verified through automated platform statistics as in this study. A cluster randomized trial in China concluded that HCPs did not use the electronic adherence data as intended, possibly due to frequent reports of pillboxes failing to log doses resulting from battery issues, although with a different platform and DAT (27). The large number of days with no facility staff accessing the adherence platform observed in South Africa and the Philippines could be explained by high workloads, or lack of motivation. A lack of trust in the adherence data, due to difficulty identifying those truly experiencing adherence issues may undermine staff motivation to use DATs (26), (30). Although qualitative data indicated use of DATs reduced workload, a finding also emphasized in other studies (28), it is possible that for HCPs already overburdened, and lacking digital skills, daily review of adherence may not have been feasible.

In both Philippines and Ukraine >50% of logins to the platform were from a smart phone rather than tablet or desktop, in contrast to the other three countries where most logins were via tablet. This may indicate that using a personal mobile phone was easier than using the facility tablet or desktop for some HCPs and aligns with a meta-analysis showing that 86% of HCP surveyed report assessing adherence using a mobile phone application (23). Accessing the platform using a mobile phone may also indicate that HCPs reviewed adherence outside of working hours, suggesting they may have been too busy during the workday. The existence of an electronic patient system as in the Philippines and Ukraine, in addition to the adherence platform may be a facilitator or a barrier to the intervention. Users of e-TB Manager in Ukraine gave high ratings to the registry’s ability to improve case management (31). TB staff were already accustomed to using digital systems, and had put in the time required to embed the behaviour change. However, they may also see it as additional work if they need to login to two separate systems for PwTB management, a finding from qualitative interviews with HCPs in Ukraine. Direct integration of adherence data into existing digital systems could mitigate this.

DAT coverage varied between countries, and the factors contributing to uptake of DATs were wide ranging. Staffing issues such as rotation in Tanzania or redeployment in Philippines, frequent power outages in South Africa, and healthcare reform as well as the war in Ukraine impacted coverage. Similar results were reported in a Ugandan trial with 52% coverage (3). Lower numbers of PwTB starting a DAT and variability over time underline the pragmatic nature of the trial where routine staff implement the intervention.

Manual dosing was an important part of the intervention, and provides a measure of HCP engagement with the intervention. The addition of manual doses to the adherence platform more than seven days after a missed dose was less than30% for both DAT types in Ethiopia, and greater than50% for both Philippines and Ukraine. HCPs would need to contact the participant to confirm if the dose was taken or not before adding a manual dose to the platform. Adding a manual dose within a short time of a missed dose may indicate a systematic approach to monitoring adherence, even if the capacity of the facility was low. Failure to add a manual dose within 1 week of a missed dose may indicate poor implementation or lack of engagement by HCP with the intervention, in addition to low capacity at the facility. It could also be due to inability to contact the PwTB. In Ukraine, the addition of manual dosing decreased significantly from pre-war to post start of the war, indicated by an increase of 7.8% to 30% of overall treatment days showing no information (32).

Overall, DAT engagement was high, measured by the percentage of digitally recorded doses. Label users recorded between 62% and 84% of doses by texting the toll-free number, conversely digital a study in Uganda where self-report of dose taking was done by calling 99DOTS was 58%, dropping further over the course of treatment (33). While a meta-analysis on DAT projects across ten countries showed that overall average adherence among PwTB with DS-TB varied between 80% and 90% (32). Engagement of pillbox users was higher than labels, with 82%-91% of doses digitally recorded. Sustained engagement offers a measure of reach of the intervention and high levels indicate users like DATs. A positive association between treatment satisfaction and medication adherence, and higher levels of treatment satisfaction among participants using a pillbox versus standard of care, was previously reported (34). Similarly, from the ASCENT trials, a study on feasibility and acceptability, showed that nearly all DAT users reported that using DATs motivated them to complete treatment (11). Furthermore, a recent scoping review suggest that video and pillbox DATs are generally acceptable with moderate to high levels of engagement (25).

The feeling of connection to their HCP was very high among participants in most countries, a finding which concurs to that of previous studies (35). It has been shown that better adherence to treatment may be facilitated by good PwTB-provider relationships (33), (36), (37) and the feeling of being care for may be enhanced by DATs. However, this was lower in Philippines, particularly for label users, (14). This is aligned with other results in Philippines, possibly indicating that HCPs had difficulty implementing the intervention as planned.