Evaluation of additional resources and stories within therapist-assisted internet-delivered cognitive behaviour therapy for alcohol misuse
Department of Psychology, University of Regina, 3737 Wascana Parkway, Regina, SK S4S 0A2, Canada
Department of Psychology, York University, 4700 Keele Street, Toronto, ON M3J 1P3, Canada
Department of Psychology, Stockholm University, 106 91 Stockholm, Sweden
Department of Clinical Neuroscience, Karolinska Institutet, Centre for Psychiatric Research, Norra Stationsgaten 69, 113 64 Stockholm, Sweden
Swiss Research Institute for Public Health and Addiction, University of Zurich, Konradstrasse 32, 8005 Zurich, Switzerland
MindSpot Clinic, MQ Health, Sydney, NSW 2109, Australia
eCentreClinic, School of Psychological Sciences, Macquarie University, Sydney, NSW 2109, Australia
Abstract
Background
Additional resources and stories are sometimes incorporated into Internet-delivered cognitive behaviour therapy (ICBT) for alcohol misuse to enhance treatment. Little is known, however, about how patients use and evaluate additional resources and stories, and how use and evaluation of additional resources and stories relates to satisfaction and outcomes.
Methods
We examined patient use and evaluation of 8 additional resources and 8 stories among 121 patients who endorsed significant alcohol misuse and were enrolled in a 6-lesson ICBT course for alcohol misuse enhanced with additional resources and stories. The additional resources addressed anger, assertiveness and communication, cannabis use, cognitive coping, grief, PTSD, sleep, and worry. Stories varied by gender, ethnicity, occupation, and severity of alcohol problems. Primary drinking outcomes included the Timeline Follow-Back (TLFB) and heavy drinking days (HDD). Diverse secondary outcomes (e.g., depression, anxiety, cravings, anger, satisfaction) were also assessed.
Results
Large within-group effects for TLFB and HDD were found. Large effects were also observed for depression and cravings, with high treatment satisfaction. 63 % of patients accessed at least one resource (M = 2.27 resources), with anger (35 %), cognitive coping (35 %), sleep (34 %) and worry (30 %) being the most used. When accessed, patients found resources informative and/or helpful to varying degrees (25–67 %). In terms of stories, 85 % of patients indicated they read the stories, and 89 % of those found them worthwhile; 65 % felt less alone and 55 % found they gave them skills to improve wellbeing. Increased use and positive ratings of additional resources were not significantly related to outcomes or satisfaction. However, positive ratings of stories were associated with confidence in managing symptoms and an interest in future treatment. Additionally, reading stories was associated with larger improvements on several secondary outcomes, including PTSD, anger, insomnia, and work and social adjustment.
Conclusions
The findings suggest that adding resources and stories to ICBT is acceptable and worthwhile. A significant number of patients reviewed these materials and found them informative and/or helpful, which suggests it is likely valuable to retain these resources for those in need. Reading stories and positive ratings of stories, rather than use and positive ratings of additional resources, was associated with increased satisfaction and some larger improvements on secondary outcomes. Further research is warranted to identify strategies that will more effectively engage patients with additional resources tailored to their specific needs.
Highlights
- •Additional resources and stories were examined in an ICBT course for alcohol misuse.
- •Resources on anger, cognitive coping, sleep, and worry were used the most.
- •Most patients read the stories and were satisfied with them.
- •Large improvements in alcohol use, depression, and cravings.
- •Adding treatment components was not detrimental to outcomes.
Article notes
Untitled section
Received 2024 Nov 19; Revised 2025 Jan 30; Accepted 2025 Feb 3; Collection date 2025 Mar.
1.Introduction
Alcohol misuse is a major contributor to the global disease burden (GBD 2020 Alcohol Collaborators, 2020). Despite this, only one in seven people with alcohol misuse receives treatment (Mekonen et al., 2020). Internet-delivered cognitive behaviour therapy (ICBT) is an evidence-based approach that helps reduce barriers to care, including stigma, a desire to manage symptoms independently, or limited access to face-to-face services (Schuler et al., 2015). ICBT typically consists of lessons or modules that teach patients CBT strategies and can either be therapist-guided or self-guided (Andersson, 2016). Common treatment components include psychoeducation on alcohol and its effects, motivational exercises, goal setting, skills training (e.g., coping with cravings, refusing drinks, problem solving, and enhancing social skills), and relapse prevention (Hadjistavropoulos et al., 2020b). A systematic review of 14 studies of ICBT for alcohol misuse found small to large improvements in alcohol-related outcomes (Hadjistavropoulos et al., 2020b). However, notable limitations to ICBT include high attrition rates (e.g., 30–50 %), and the exclusion of patients with psychiatric comorbidity (Hadjistavropoulos et al., 2020b). These findings indicate a need for improvement in patient engagement and attention to comorbidities within ICBT programs for alcohol misuse.
The perceived helpfulness of treatment has been identified as an important factor in whether individuals endorsing alcohol misuse seek treatment and is also associated with treatment outcomes (Degenhardt et al., 2021). Therefore, improving the perceived helpfulness of treatment for alcohol misuse could serve to increase treatment uptake, engagement, and outcomes. In recent years, gathering feedback from interested parties has been recommended when developing and refining interventions. The UK Medical Research Council (Skivington et al., 2021) emphasizes that engaging diverse stakeholders, including those with personal or professional interests, can maximize the effectiveness and real-world use of interventions. Supporting this view, a systematic review of 66 studies (Brett et al., 2014) found that patient and public involvement improved the quality and relevance of health and social research across all stages, such as development of user-friendly materials, recruitment strategies, data interpretation, and dissemination. Additionally, other research has shown that addressing patient preferences enhances engagement with interventions and patient outcomes (Preference Collaborative Review Group, 2008).
With the above research in mind, our clinic, which offers ICBT for alcohol misuse in routine care, formed a steering committee to identify opportunities to enhance ICBT for alcohol misuse (Hadjistavropoulos et al., 2021a). While several opportunities were identified (Hadjistavropoulos et al., 2021a), two directions were deemed particularly promising for improving engagement and outcomes of ICBT for alcohol misuse: 1) addressing comorbid concerns (i.e., anger, assertiveness and communication, cannabis use, cognitive coping skills, grief, post-traumatic stress disorder (PTSD), sleep, and worry); and 2) increasing the diversity of patient stories that are included in ICBT for alcohol misuse so they show greater variability in gender, ethnicity, occupation and severity of alcohol problems. Adding resources and stories allows patients to conveniently and efficiently tailor treatment to their specific needs. Research on additional resources in ICBT for other mental health concerns has found a small but positive association between ratings of additional resources and ICBT satisfaction, as well as slightly larger reductions in symptoms of depression, anxiety, PTSD, and insomnia (Hadjistavropoulos et al., 2024). Similarly, patient satisfaction with stories contained in ICBT has been found to be associated with overall ICBT treatment satisfaction and reduced anxiety severity (Hadjistavropoulos et al., 2023). The promising findings suggest that further study of additional resources and patient stories within ICBT for alcohol misuse is worth investigating and could inform program optimization efforts.
1.1.Objectives
This study evaluated the inclusion of enhanced content in ICBT for alcohol misuse, including new additional resources and diverse stories. The intervention, entitled the Alcohol Change Course Enhanced (ACCE), was examined through the following research questions: 1) How does the ACCE affect patients' weekly alcohol consumption and number of heavy drinking days (HDD)?; 2) What is the impact of the ACCE on secondary outcomes (e.g., depression, anxiety, anger, cravings) and treatment satisfaction?; 3) How are additional resources used and rated by patients, and how do these ratings relate to treatment satisfaction and outcomes?; and 4) How are patient stories used and rated by patients, and how do these ratings relate to treatment satisfaction and outcomes?
2.Methods
2.1.Setting and ethics
This research on the ACCE was conducted by the Online Therapy Unit (OTU) which is based at the University of Regina and funded by the Saskatchewan Government (Ministry of Health) to provide free ICBT to Saskatchewan residents as part of public health care (Hadjistavropoulos et al., 2014; Hadjistavropoulos et al., 2021b). Patients learn about the OTU in the same way they learn about other public health services, with the three most common ways being through a physician or other health professional, through OTU advertising (e.g., online, print), or through a family, friend, or coworker. The trial was registered with ClinicalTrials.gov (NCT05555264) and received ethics approval from the University of Regina Research Ethics Board (file # 2022–082). All patients provided informed consent for their participation in the ACCE and the associated research.
2.2.Eligibility criteria
To participate, patients first completed an online screening questionnaire, followed by a telephone screening to confirm eligibility. Specifically, to be eligible for this study, prospective patients had to meet the following inclusion criteria: be at least 18 years old, reside and will be in Saskatchewan during the treatment period, have access to and comfort using a computer and the Internet, and endorse alcohol misuse defined as a score ≥ 8 on the Alcohol Use Disorders Identification Test (Saunders et al., 1993) and consuming ≥14 drinks in the preceding week (i.e., cut-offs used in similar previous research; Sundström et al., 2020). Exclusion criteria included: having a severe medical condition and/or psychiatric condition (e.g., unmanaged psychosis, severe depression as suggested by (Patient Health Questionnaire – 9 (PHQ-9) ≥ 24 and clinical interview; Kroenke et al., 2001), high suicide risk (assessed by item 9 of PHQ-9 and in a clinical interview), severe substance use problems other than alcohol and/or cannabis (defined as >24 on the Drug Use Disorders Identification Test but also discussion with a clinician during intake; Berman et al., 2002), severe cognitive impairment, low motivation or concerns about completing ICBT as indicated by the patient during the interview, or currently receiving or expecting to receive individual alcohol treatment more than twice per month. Notably, the ACCE was available to Saskatchewan residents who did not meet the alcohol consumption criteria; however, these clients were not included in the current study, as the focus was on understanding stories and additional resources among individuals with significant alcohol concerns. The ACCE was also accessible to non-Saskatchewan residents, but only in a self-guided format, as therapist-guided support is funded by the Saskatchewan government. Due to these differing circumstances regarding treatment offerings, non-Saskatchewan clients were also excluded from this study.
2.3.Measures1
2.3.1.Primary outcome measures
Measures of drinks in the previous week and HDD were administered at pre-treatment, mid-treatment (week 4), post-treatment (7 weeks; expected period when clients would have reviewed 6 core lessons), and at 4-month follow-up. Questionnaires about the additional resources and stories were administered at weeks 4 and 8 to allow patients additional time to review the resources before completing these questionnaires.
Consumption in the previous week
The Timeline Followback (TLFB; Sobell and Sobell, 1992) was used to assess for alcohol consumption in the previous week. Patients reported on the total number of standard drinks of alcohol (i.e., 4.5 oz. glass of 12 % wine; 1.3–1.5 oz. shot of 40 % liquor; or 12 oz. of 5 % beer, cider, or cooler) they consumed on each day in the preceding week. Cronbach's α for the TLFB ranged from 0.75 to 0.89 in this study.
Heavy drinking days
Data from the TLFB was also used to calculate the number of HDD in the previous week, which were defined as days in which women consumed more than three standard drinks and men consumed more than four standard drinks (Sobell and Sobell, 1992).
Additional Resources Questionnaire (ARQ). The ARQ consisted of 4 items and was developed for the current trial. Patients were asked which of the 8 additional resources (i.e., anger, assertiveness and communication, cannabis use, cognitive coping, grief, PTSD, sleep, worry) they found informative, helpful for their alcohol use, and helpful for their overall wellbeing. The questionnaire also included an open-ended question about suggestions for new additional resources that should be developed. The online platform also tracked whether the additional resources were accessed. While the ARQ was administered at weeks 4 and 8, some patients responded at only one of the time points. Therefore, we combined the 4 and 8 week responses for each additional resource using the following algorithm: if there was a response at either 4 or 8 weeks, the additional resource variable was coded as 1; otherwise, it was coded as 0. Furthermore, for ease of interpretation, ratings of informativeness, helpfulness for alcohol use, and helpfulness for general well-being were summed and recoded as follows: if the sum = 0, it was recoded as 0 (indicating that the additional resource was neither informative nor helpful), and if the sum was ≥1, it was recoded as 1 (indicating that the additional resource was rated as informative and/or helpful). We then created a composite score summing the number of resources accessed and the number of resources that were rated as informative and/or helpful.
Patient Stories Questionnaire. The Patient Stories Questionnaire, developed for this trial, consisted of 8 items and asked patients about their impressions of stories overall (described as reflections). Patients first responded to an item that stated “I reviewed the reflections”, and rated it on a scale of 1 (“Did not read any of them”) to 5 (“Read them all”). If patients selected a rating of 2 or higher on the first item, they were presented with an additional 7 items. Five of the items were rated on a scale of 1 (“Strongly disagree”) to 5 (“Strongly agree”) and asked patients about various features of the resources (i.e., relatability, feeling less alone in their mental health experiences, increased knowledge about mental health, provided ideas for skills use, and motivated them to use the skills). The five Likert-type items were summed to create a composite variable with total score ranging from 5 to 25. Patients also respond to a “Yes”/“No” question about whether reading the reflections was worth their time and to an open-ended question about whether they had any suggestions for improving the reflections.
2.3.2.Secondary measures
Secondary measures were administered at screening, week 7 (post-treatment), and 4-month follow-up, unless otherwise indicated.
Patient Health Questionnaire 9-Item (PHQ-9;Kroenke et al., 2001). The PHQ-9 is a self-report measure assessing depressive symptoms in the previous 2-week period. Scores range from 0 to 27, with higher scores indicating more severe symptoms, and a score of ≥10 suggesting clinically significant level of depression. In the current study, Cronbach's α for the PHQ-9 ranged from 0.85 to 0.86. Of note, this measure was also given at week 4 for clinical purposes.
Generalized Anxiety Disorder 7-Item (GAD-7;Spitzer et al., 2006). The GAD-7 consists of 7 self-report items assessing symptoms of generalized anxiety in the previous 2-week period. Scores range from 0 to 21, with higher scores indicating more severe symptoms of anxiety. A score of ≥10 suggests a clinically significant level of anxiety. Cronbach's α for the GAD-9 ranged from 0.89 to 0.90 in this study.
Alcohol Use Disorder Identification Test (AUDIT;Saunders et al., 1993). The AUDIT is a 10-item validated measure used to screen for risky or hazardous alcohol consumption. Scores range from 0 to 40. Scores between 8 and 14 suggest hazardous or harmful alcohol consumption and scores of ≥15 indicate likely alcohol dependence or a moderate to severe alcohol use disorder. Cronbach's α for the AUDIT ranged from 0.73 to 0.82 in this study.
Penn Alcohol Craving Scale (PACS;Flannery et al., 1999). The PACS includes 5 self-report items that focus on the frequency, intensity, and duration of alcohol craving, the patient's ability to resist drinking alcohol, and their overall craving for alcohol in the previous week. Scores range from 0 to 30, with higher scores indicating greater craving. Cronbach's α for the PACS ranged from 0.86 to 0.91 in this study.
Work and Social Adjustment Scale (WSAS;Mundt et al., 2002). The WSAS is a 5-item self-report measure that focuses on impairment in functioning, with total scores ranging from 0 to 40. In the current study, Cronbach's α for the WSAS ranged from 0.85 to 0.92.
Dimensions of Anger Reactions (DAR-5;Forbes et al., 2014). The DAR-5 includes 5 self-report items assessing concerns with anger over the past 4 weeks. Total scores range from 0 to 25 and a score of ≥12 suggests that further assessment and treatment of anger may be warranted. Cronbach's α for the DAR ranged from 0.78 to 0.84 in this study.
Posttraumatic Stress Disorder Checklist for the DSM-5 (PCL-5;Blevins et al., 2015) consisting of 20 items assessing for symptoms of PTSD rated on a 0 to 4 scale was administered at pre-treatment. A cut-off score of >32 suggests a likely PTSD diagnosis (Weathers et al., 2013). In the current study, Cronbach's α for the PCL-5 was 0.93.
Short-Form Posttraumatic Stress Disorder Checklist for DSM-5 (SF-PCL-5;Zuromski et al., 2019) was administered at post-treatment and follow-up. The SF-PCL-5 includes four items to assess for symptoms of PTSD rated on a 0 to 4 scale with a possible score of 0–16. Its psychometric properties are comparable to the longer PCL-5. A score of ≥10 suggests a likely PTSD diagnosis (Price et al., 2016). In the current study, Cronbach's α for the SF-PCL-5 ranged from 0.74 to 0.81. For longitudinal comparisons, items from the PCL-5 were used to create a pre-treatment SF-PCL-5 score.
Insomnia Severity Index (ISI;Morin et al., 2011). The ISI includes 7 self-report items assessing difficulties falling asleep, staying asleep, and waking up too early. Scores range from 0 to 28 and scores of >9 have been used to detect cases of insomnia. In the current study, Cronbach's α for the ISI ranged from 0.89 to 0.92.
Cannabis Use Disorder Identification Test – Revised (CUDIT-R;Adamson et al., 2010). The CUDIT-8 includes 8 items to screen for cannabis-related problems, with total scores ranging from 0 to 32. A score > 7 suggests hazardous cannabis use and a score > 11 indicates a possible cannabis use disorder. In the current study, Cronbach's α for the CUDIT-R ranged from 0.84 to 0.88.
Couples Satisfaction Index 4-Item (CSI-4;Funk and Rogge, 2007). The CSI-4 includes 4 items measuring satisfaction within a romantic relationship. Scores range from 0 to 21, with higher scores indicating greater relationship satisfaction. A score < 13.5 suggests relationship dissatisfaction. Cronbach's α for the CSI ranged from 0.91 to 0.97 in this study.
Motivation for Change (MFC). Patients' motivation for change was assessed using three items developed for this study: 1) How important is it to you to change your alcohol use?, 2) How confident are you that you can change your alcohol use?, and 3) How ready are you to change your alcohol use now? Each item was scored on a 10-point scale, ranging from 1 = Not important/not confident/not ready to 10 = Very important/very confident/very ready. The three items were summed to obtain a total score ranging from 3 to 30. Cronbach's α for the MFC ranged from 0.61 to 0.78 in this study.
Credibility and Expectancy Questionnaire (CEQ;Devilly and Borkovec, 2000). At mid-treatment, patients' perceptions of treatment credibility and expectations of success were measured using the 6-item Credibility and Expectancy Questionnaire (CEQ; Devilly and Borkovec, 2000). For the first three items, patients rated each item on a 9-point scale (1 = not at all logical/useful/confident, 9 = very logical/useful/confident), and these were summed to create a total score between 3 and 27, with higher scores reflecting greater credibility (credibility subscale of CEQ; Cronbach's α = 0.85). For the last three items, items 4 and 6 were rated on an 11-point scale (0–100, recoded as 0–10), while item 5 used a 9-point scale (1–9). These items were summed to generate a total expectancy score between 1 and 29, with higher scores indicating greater treatment expectancy (expectancy subscale of CEQ; Cronbach's α = 0.84).
2.4.Alcohol change course enhanced (ACCE)
The course was originally developed in Switzerland (Baumgartner et al., 2021; Schaub et al., 2016) and prior to the current study, was translated into English and studied with young adults (Frohlich et al., 2021) before undergoing two rounds of adaptations by an OTU Patient-Oriented Research Steering Committee (PORSC) to tailor the course to the local context. The PORSC includes clinicians, healthcare managers, researchers, and people-with-lived-experiences (some members have changed over time). In terms of process, at PORSC meetings, the OTU team would present information to the committee and committee members would then either provide feedback during the meeting, via email, or during subsequent one-on-one meetings. The committee would then make recommendations for improvement, which the OTU would act upon between meetings. While the core course content continues to focus on methods to reduce or eliminate alcohol use by changing habits, the first adaptation of the course drew on interviews with interested parties (Hadjistavropoulos et al., 2020a) and incorporated alcohol-related education (e.g., Canadian prevalence, guidelines, and physical health impacts). Additionally, the materials were adapted to the OTU practices releasing the 12 lessons with accompanying worksheets consecutively over 8 weeks. In the second adaptation of the course, based on user feedback (Hadjistavropoulos et al., 2021a), the PORSC made the language more inclusive, and condensed the content from 12 to 8 lessons, one per week. Both these versions were found to show significant large within-group effects on the TLFB and HDD (Sundström et al., 2022; Sapkota et al., 2024). Nevertheless, after reviewing patient feedback (Hadjistavropoulos et al., 2021a), the PORSC recommended some further improvements to the course. In the version of the course used in this study, the same alcohol-related content was condensed and delivered in six consecutive online lessons addressing: a) goal setting, b) identifying risk situations, and seemingly unimportant decisions, c) learning to say no and cope with cravings, d) problem solving skills, e) thought challenging and f) managing relapses and preserving success. Each lesson focused on alcohol-related content but also provided some attention to depression, anxiety and stress throughout. Each lesson included psychoeducational slides and downloadable worksheets to practice skills. Each lesson was designed to be completed in one week, but during week 4 and 8, clients were encouraged by their therapists to review and work on materials that were previously distributed and to review additional resources of interest to them. Based on PORSC feedback, 8 resources were added to the course addressing comorbid concerns that were not elaborated on within the course to a significant degree (i.e., anger, assertiveness and communication, cannabis use, cognitive coping for managing emotions, grief, PTSD, sleep, and worry). The resources were not part of the original course and ranged from 7 to 15 pages (2919–5172 words) and were accessible at any point during the course in a designated “Additional Resources” section. The committee also recommended enhancing stories in the course, increasing the number of patient stories from 6 to 8 and modifying them so that there was greater variation in alcohol use severity, gender, ethnicity, age, and community size. The stories were integrated in each of the six lessons (2 stories per slide) and referred to as personal reflections. The stories in the first lesson introduced the characters' demographics, their adverse life circumstances, and the challenges they faced, including problems with alcohol use and associated cognitive, behavioural, and physical symptoms. The stories captured how each lesson's core skill was implemented by the characters and also their successes and challenges. This was done intentionally to reflect the authentic experience of changing alcohol use and normalizing the imperfections involved. The purpose of the stories was to provide relatable, first-person perspectives to subtly encourage behavioural change and reassure patients that they were are not alone in their experiences.
2.5.Therapist support
At enrollment, patients could choose between two options: completing the course independently (self-guided monitored group) or with weekly therapist support through secure messages or phone calls if clinically indicated (standard group). In both groups, weekly automated emails were sent to encourage consistent engagement with the lessons. Also, ICBT therapists monitored patient progress and weekly questionnaires completed by patients to track alcohol use and symptoms. Patients in the self-guided group typically had no direct contact with therapists but would receive a phone call if there was a sudden increase in alcohol use or depression symptoms to ensure patient safety and provide guidance as needed. Similarly, patients in the standard group were also called if such changes occurred. Additionally, standard group patients received a weekly message from their therapist on a designated day responding to any information, questions or concerns shared by patients during the previous week. These messages generally took 15 min to compose and offered support, provided additional guidance on course content, and encouraged patient participation in the course.
2.6.Statistical analyses
Analyses were conducted using IBM SPSS Statistics (Version 29.0). Descriptive statistics, including percentages, means, and standard deviations were used to summarize patient characteristics. To assess changes over time in the two primary outcomes (i.e., TLFB, HDD) and secondary outcomes (e.g., GAD-7, PHQ-9, PACS, DAR-5), a series of mixed-effects models were employed. All available data from four assessment points (pre-treatment, mid-treatment [week 4], post-treatment [week 7], and follow-up [week 20]) were used for the primary outcomes, and three assessment points (pre-treatment, post-treatment [week 7], and follow-up [week 20]) for the secondary outcomes. Each outcome was analyzed using models that incorporated fixed and random effects to account for both the intercept (pre-treatment symptom scores) and the slope (time), addressing the correlated nature of the data. Intraclass correlation coefficients were evaluated to confirm the appropriateness of the mixed-effects models (Peugh and Enders, 2005).
We tested several within-subject covariance structures (e.g., unstructured, autoregressive [AR(1)], AR(1): heterogeneous) to identify the best fit for our data. The models with the lowest Akaike's Information Criterion (AIC) and Bayesian Information Criterion (BIC) were selected for the final analysis. For most outcomes, the first-order autoregressive [AR(1)] structure provided the best fit. However, for TLFB, the AR(1): heterogeneous structure fit the data better. Estimates were generated using the full information maximum likelihood method, with the Satterthwaite approximation applied to determine the denominator degrees of freedom. Cohen's d effect sizes (Cohen, 1988), for the differences between pre- and post-treatment, and pre-treatment and follow-up, were computed using estimated marginal means and standard deviations from the mixed-model analysis, along with the associated 95 % confidence intervals for each outcome.
Engagement, satisfaction, additional resource use and ratings, and stories use and ratings were examined through descriptive statistics. Number of additional resources used and rated as informative and/or helpful and use and ratings of stories were examined for associations with satisfaction questions and treatment outcomes on both primary and outcome measures (i.e., pre-to-post changes) using correlations (Pearson's r, point-biserial correlation [rpb], or phi, as appropriate).
2.6.1.Missing data management
There were no missing values across pre-treatment variables, but 30 (24.8 %) and 36 patients (29.8 %) had missing data for primary outcome measures at mid- and post-treatment, respectively. Up to 40 patients (33.1 %) had missing data on secondary measures at post-treatment, except for the CUDIT-R and SF-PCL-5 measures. At pre-treatment, only 47 patients (38.8 % of the total sample of 121) completed the CUDIT-R measure, and 77 (63.6 % of the sample) completed the SF-PCL-5. At the 3-month follow-up, data were missing for up to 76 patients (62.8 %) due to non-completion. An analysis of missingness using Little's Missing Completely at Random test (χ2 = 1888.73, df = 2002, p = .96) suggested that the data were missing at random (Little and Rubin, 2020). Because longitudinal mixed-model analysis with the maximum likelihood estimation method can handle missing data, imputation prior to analysis was not necessary (Twisk et al., 2013). However, following modified intention-to-treat (ITT) analyses, and as part of sensitivity analyses, thirty multiply imputed datasets were created (Enders, 2022; Graham et al., 2007) and analyzed for the primary outcomes (i.e., TLFB and HDD). For the imputation, all demographic variables and baseline assessments of two of the primary outcomes were included as predictors.
To control for the potential confounding effects of missing data, demographic variables that significantly predicted missing outcomes at post-treatment (e.g., age) were included as covariates in the final longitudinal-mixed model analyses. Additionally, clinically relevant variables—such as the number of lessons completed, treatment expectancy and credibility, years of alcohol use, level of motivation, concurrent mental health treatments, other diagnosed mental disorders, and family history of alcohol use—were also controlled by including them as covariates in the final models. Given the high percentage of missing data in the primary outcomes (62.8 %) at follow-up, we conducted a mixed-model analysis using data from only three time points (pre, mid [4 weeks], and post [7 weeks]) and compared the results with those obtained from the four assessment time points.
2.6.2.Qualitative analysis
Patient responses to open-ended questions about the additional resources and stories were imported into Excel. Responses were coded using conventional content analysis (Hsieh and Shannon, 2005), which included identifying relevant themes that could be quantified. One coder (EV) identified themes, which were then reviewed by the primary investigator in the study (HDH).
3.Results
3.1.Pre-treatment sample and characteristics
As shown in Fig. 1, 451 individuals completed the online screening for the ACCE from September 16, 2022 to June 19, 2024. Of these, 121 patients from Saskatchewan (SK) scored ≥ 14 on TLFB and ≥ 8 on AUDIT and were included in the current study as per the trial registration. As in our previous preference trial (Sapkota et al., 2024), only a small number of patients (n = 21/121; 17.4 %) chose the self-guided ACCE, while the majority (n = 100/121; 82.6 %) selected therapist guidance. These groups did not differ significantly in assessed demographic variables, except for gender (p < .01), where more women chose the therapist-guided option (men: n = 43/100; 43 % vs. women: n = 57/100; 57 %), while more men selected the self-guided option (men: n = 15/20; 75 % vs. women: n = 5/20; 25 %; one individual did not disclose their gender identity). Additionally, there were no significant differences in pre-treatment primary outcome variables (i.e., TLFB and HDD). Since past research shows no differences in outcomes between these two groups (Sapkota et al., 2024), we present only the overall results (i.e., not comparing therapist-guided and self-guided groups) in this study.
The descriptive statistics for ACCE patients' demographic and clinically relevant characteristics are shown in Table 1. The patients were primarily in middle adulthood (M = 45 years old, SD = 11.97, range = 22–73 years old), women (51.2 %), educated beyond high school (76.8 %), White (86 %), married (61.2 %), employed in paid work (77.7 %), and residing in a large city (52.9 %). A significant proportion (54.6 %) had been struggling with alcohol-related problems for six or more years, had a family history of alcohol use (94.2 %), while 47.1 % reported having been diagnosed with a mental disorder, and 38.8 % reported currently receiving some other treatment (e.g., regular medical appointments). Patient scores indicated significant comorbid concerns, with 65.3 % reporting sleep difficulties, 61.2 % elevated depression, 47.9 % elevated anxiety, 43.0 % relationship discord, 26.4 % symptoms of PTSD, 21.5 % anger issues, and 16.6 % cannabis use problems,
| M | SD | |
|---|---|---|
| Age (range: 22–73) | 45 | 11.97 |
| n | % | |
| Gender | ||
| Men | 58 | 47.9 |
| Women | 62 | 51.2 |
| Other, prefer not to disclose | 1 | 0.8 |
| Education | ||
| High school diploma or less | 28 | 23.1 |
| Some college or university | 51 | 42.1 |
| University degree | 42 | 34.7 |
| Ethnicity | ||
| White/Caucasian | 104 | 86.0 |
| Indigenous | 14 | 11.6 |
| Other | 3 | 2.5 |
| Relationship status | ||
| Never married single/dating | 26 | 21.5 |
| Married/living with a partner | 74 | 61.2 |
| Separated/divorced/widowed | 21 | 17.4 |
| Do you have paid work? | ||
| Yes | 94 | 77.7 |
| No | 27 | 22.3 |
| Location | ||
| Large City (population over 200,000) | 64 | 52.9 |
| Small to Medium City (population of 10,000 to 200,000) | 24 | 19.8 |
| Town, village or reserve | 33 | 27.3 |
| How did you hear about OTU? | ||
| Advertisement (e.g., posters, radio, TV, website, email) | 31 | 25.6 |
| Friend/family/employer | 26 | 21.5 |
| Health professionals | 52 | 43.0 |
| Other | 12 | 9.9 |
| Years with alcohol problems | ||
| 0–2 years | 22 | 18.2 |
| 3–5 years | 33 | 27.3 |
| 6–10 years | 26 | 21.5 |
| >10 years | 40 | 33.1 |
| Family history of alcohol | ||
| Yes | 114 | 94.2 |
| No | 7 | 5.8 |
| Alcohol history in first degree relatives (e.g., biological father, mother, siblings) | ||
| Yes | 97 | 80.2 |
| No | 24 | 19.8 |
| Alcohol history in second degree relatives (e.g., grand parents, uncle, aunts, cousins) | ||
| Yes | 100 | 82.6 |
| No | 21 | 17.4 |
| Self-report diagnosis of mental disorders? | ||
| Yes | 57 | 47.1 |
| No | 64 | 52.9 |
| Currently receiving other mental health treatment? | ||
| Yes | 47 | 38.8 |
| No | 74 | 61.2 |
| PHQ-9 > 9 | 74 | 61.2 |
| GAD-7 > 9 | 58 | 47.9 |
| DAR > 11 | 26 | 21.5 |
| PCL-5 > 32 | 35 | 26.4 |
| ISI > 9 | 79 | 65.3 |
| CUDIT > 7 | 20 | 16.6 |
| CSI-4 < 13.5 | 52 | 43.0 |
3.2.Primary outcomes
Pre-treatment to follow-up means, standard deviations, and effect sizes for the primary outcomes can be found in Table 2. The mixed-model analysis revealed a significant effect of time on the reduction of TLFB scores (β = −3.32, SE = 0.54, p < .001), with a large pre-to-post Cohen's effect size of d = 1.31 (95 % CI [1.00, 1.61]). The time effect remained significant (p = .002) after controlling for demographic factors and clinically relevant variables. Among the covariates, level of pre-treatment motivation was also statistically significant, indicating that patients with higher pre-treatment motivation showed greater improvement in the number of standard weekly drinks with treatment (β = −2.96, SE = 0.98, p = .003).
| Primary outcomes | n | Mean | SD | d | 95%CI |
|---|---|---|---|---|---|
| TLFB | |||||
| Pre | 121 | 41.79 | 23.65 | ||
| Mid | 91 | 21.64 | 22.74 | 0.87 | 0.58–1.15 |
| Post | 85 | 14.67 | 15.80 | 1.31 | 1.0–1.61 |
| Follow-up | 46 | 15.24 | 13.04 | 1.25 | 0.88–1.61 |
| HDD | |||||
| Pre | 120 | 4.43 | 2.29 | ||
| Mid | 90 | 2.38 | 2.20 | 0.91 | 0.62–1.19 |
| Post | 85 | 1.72 | 2.21 | 1.20 | 0.90–1.50 |
| Follow-up | 45 | 1.97 | 2.08 | 1.10 | 0.73–1.46 |
| Secondary outcomes | n | Mean | SD | d | 95%CI |
| PHQ-9 | |||||
| Pre | 121 | 11.86 | 5.28 | ||
| Post | 83 | 6.57 | 4.91 | 1.03 | 0.73–1.33 |
| Follow-up | 46 | 5.73 | 4.65 | 1.20 | 0.83–1.56 |
| GAD-7 | |||||
| Pre | 121 | 9.51 | 5.28 | ||
| Post | 83 | 6.39 | 4.93 | 0.61 | 0.32–0.89 |
| Follow-up | 46 | 5.74 | 4.56 | 0.74 | 0.39–1.10 |
| AUDIT | |||||
| Pre | 121 | 23.83 | 6.74 | ||
| Post | 67 | 20.44 | 5.84 | 0.53 | 0.22–0.83 |
| Follow-up | 36 | 18.32 | 5.43 | 0.85 | 0.47–1.23 |
| PACS | |||||
| Pre | 121 | 18.69 | 6.30 | ||
| Post | 83 | 13.82 | 6.02 | 0.79 | 0.50–1.08 |
| Follow-up | 46 | 11.27 | 5.81 | 1.20 | 0.84–1.57 |
| WSAS | |||||
| Pre | 121 | 14.28 | 8.99 | ||
| Post | 81 | 10.37 | 8.46 | 0.45 | 0.16–0.73 |
| Follow-up | 19 | 7.69 | 7.58 | 0.75 | 0.26–1.24 |
| DAR-5 | |||||
| Pre | 121 | 9.33 | 3.21 | ||
| Post | 83 | 7.99 | 3.04 | 0.43 | 0.14–0.71 |
| Follow-up | 44 | 7.59 | 2.90 | 0.56 | 0.21–0.91 |
| SF-PCL-5 | |||||
| Pre | 77 | 4.98 | 3.10 | ||
| Post | 81 | 3.69 | 3.11 | 0.41 | 0.10–0.73 |
| Follow-up | 44 | 3.53 | 2.85 | 0.48 | 0.11–0.86 |
| ISI | |||||
| Pre | 121 | 12.04 | 5.89 | ||
| Post | 81 | 10.43 | 5.54 | 0.28 | n.s. |
| Follow-up | 44 | 8.39 | 5.23 | 0.64 | 0.29–0.99 |
| CUDIT-R | |||||
| Pre | 47 | 7.56 | 5.86 | ||
| Post | 24 | 6.19 | 4.76 | 0.25 | n.s. |
| Follow-up | 12 | 8.14 | 4.15 | −0.10 | n.s. |
| CSI-4 | |||||
| Pre | 82 | 11.76 | 4.74 | ||
| Post | 63 | 12.13 | 4.34 | 0.08 | n.s. |
| Follow-up | 30 | 12.34 | 3.60 | 0.13 | n.s. |
| MFC | |||||
| Pre | 121 | 25.52 | 4.17 | ||
| Post | 89 | 24.83 | 3.98 | 0.17 | n.s. |
| Follow-up | 81 | 25.18 | 3.95 | 0.08 | n.s. |
Similarly, the mixed-model analysis showed a significant effect of time on the reduction of HDD scores (β = −0.49, SE = 0.08, p < .001), with a large pre-to-post Cohen's effect size of d = 1.20 (95 % CI [0.90, 1.50]). The time effect remained significant (p < .001) after controlling for demographic and clinically relevant variables. Among the covariates, level of pre-treatment motivation was again statistically significant (β = −0.39, SE = 0.12, p = .002), indicating that patients with higher pre-treatment motivation showed greater improvement in the number of weekly HDD over time. Additionally, age (β = 0.03, SE = 0.01, p = .04), gender (β = 0.78, SE = 0.35, p = .03), and number of years with alcohol use problems (β = 0.34, SE = 0.16, p = .04) were also statistically significant covariates. Namely, older patients, women, and those with a longer history of alcohol use were less likely to improve in the HDD over times.
Similar results were obtained for the effect of time when analyzing the multiply-imputed data with four assessment points for TLFB (β = −2.90, SE = 0.81, p < .001) and HDD (β = −0.35, SE = 0.08, p < .001), as well as when using three assessment points (pre-treatment, mid-treatment, and post-treatment) for TLFB (β = −3.83, SE = 0.33, p < .001) and HDD (β = −0.40, SE = 0.04, p < .001).
3.3.Secondary outcomes
Pre-treatment to follow-up means, standard deviations, and effect sizes for the secondary outcomes are also presented in Table 2. The mixed-model analyses for the secondary outcome variables revealed a significant effect of time on patients' scores for the following: PHQ-9 (β = −1.32, SE = 0.15, p < .001), GAD-7 (β = −0.80, SE = 0.13, p < .001), AUDIT (β = −1.16, SE = 0.17, p < .001), PACS (β = −1.54, SE = 0.18, p < .001), WSAS (β = −1.58, SE = 0.33, p < .001), DAR-5 (β = −0.37, SE = 0.09, p < .001), SF-PCL-5 (β = −0.27, SE = 0.09, p = .003), and ISI (β = −0.74, SE = 0.16, p < .001). However, no significant effect of time was found for the CUDIT-R (β = 0.04, SE = 0.21, p = .85), CSI-4 (β = 0.12, SE = 0.11, p = .28), and MFC (β = −0.05, SE = 0.09, p = .53).
3.4.Treatment engagement and satisfaction
A total of 51.3 % of patients completed all six lessons, with a mean (SD) of 4.45 (1.92) lessons completed. Overall, 67.8 % of patients completed four or more lessons. A total of 81 (66.9 %) patients responded to the satisfaction questions. As shown in Table 3, 81.5 % of patients were satisfied with the treatment overall, 87.7 % were satisfied with the course materials, 92.6 % felt the course was worth their time, and 95.1 % said they would recommend it to a friend. A large majority reported that taking the course increased their confidence in managing alcohol use (80.2 %) and their motivation to seek other treatments if needed in the future (84 %).
| Treatment ratings | n | % |
|---|---|---|
| Satisfied overall with treatment | ||
| Satisfied | 66 | 81.5 |
| Neutral | 14 | 17.3 |
| Dissatisfied | 1 | 1.2 |
| Satisfied with materials | ||
| Satisfied | 71 | 87.7 |
| Neutral | 9 | 11.1 |
| Dissatisfied | 1 | 1.2 |
| Confidence in managing symptoms | ||
| Increased | 65 | 80.2 |
| No Change | 11 | 13.6 |
| Increased | 5 | 6.2 |
| Motivation to seek other treatment | ||
| Increased | 68 | 84.0 |
| No Change | 11 | 13.6 |
| Reduced | 2 | 2.5 |
| Course was worth their time | ||
| Yes | 75 | 92.6 |
| No | 6 | 7.4 |
| Would recommend the course to a friend | ||
| Yes | 77 | 95.1 |
| No | 4 | 4.9 |
| Engagement | ||
| Completed majority of lessons (i.e. 4 or more) | 82 | 67.8 |
| Completed all lessons | 62 | 51.2 |
| M | SD | |
| Lessons completed | 4.45 | 1.92 |
3.5.Additional resources
3.5.1.Use and evaluation
Table 4 presents the percentages for each of the additional resources accessed, as well as ratings of informativeness and/or helpfulness. Of the 121 patients, 62.8 % accessed at least one additional resource, while 37.2 % did not access any additional resources. Patients accessed an average of 2.27 resources (SD = 2.64) out of 8 resources. The most frequently accessed resources addressed anger (34.7 %), cognitive coping (34.7 %), sleep (33.9 %), and worry (29.8 %). Ratings of perceived informativeness and/or helpfulness ranged from 25 % (cannabis use) to 66.7 % (worry). The resources which were perceived to be most informative/helpful were on worry (66.7 %), anger (61.9 %), cognitive coping (57.1 %) and grief (57.1 %).
| Additional resource topics | Accessed out of 121 | Informative and/or Helpful | ||
|---|---|---|---|---|
| n | % | n | % | |
| Anger | 42 | 34.7 | 26 | 61.9 |
| Assertiveness and communication | 32 | 26.4 | 13 | 40.6 |
| Cannabis use | 24 | 19.8 | 6 | 25.0 |
| Cognitive coping | 42 | 34.7 | 24 | 57.1 |
| Grief | 28 | 23.1 | 16 | 57.1 |
| Post-traumatic stress disorder | 30 | 24.8 | 12 | 40.0 |
| Sleep | 41 | 33.9 | 19 | 46.3 |
| Worry | 36 | 29.8 | 24 | 66.7 |
| Accessed at least one additional resource | n | % | ||
| No | 45 | 37.2 | ||
| Yes | 76 | 62.8 | ||
| Accessed additional resource | M | SD | ||
| 2.27 | 2.64 | |||
We conducted a secondary analysis to examine the percentage of patients with clinically elevated scores on certain measures who accessed resources on related issues. These results are reported in Table 5. While most patients with clinical scores did not access the resources a significant number did, with 50 % of patients with likely problems with cannabis accessing the cannabis resource, 42.3 % of patients with anger accessing the anger resource, 40.5 % of patients with elevated insomnia scores accessing the sleep resource, 37.8 % of patients with depression accessing the cognitive coping resource, and 34.2 % of patients with elevated PTSD symptoms accessing the PTSD resource. Patients who scored above the clinical cut-off for the corresponding resource were most likely to rate the anger (90.9 %, n = 10/11), worry (72.2 %, n = 13/18), and grief resources (64.7 %, n = 11/17) as informative and/or helpful. The resources that were least likely to be rated as informative and/or helpful among patients who scored above the corresponding clinical cut-off were the cannabis use (40.0 %, n = 4/10) and the assertiveness and communication resource (41.2 %, n = 7/17).
| Additional resource topics | Outcome measures above the clinical cut-off (n) | Accessed | Found informative and/or helpful | ||
|---|---|---|---|---|---|
| n | % | n | % | ||
| Anger | DAR-5 > 11 (n = 26) | 11 | 42.3 | 10 | 90.9 |
| Assertiveness and communication | CSI-4 < 13.5 (n = 52) | 17 | 32.7 | 7 | 41.2 |
| Cannabis use | CUDIT-R > 7 (n = 20) | 10 | 50.0 | 4 | 40.0 |
| Grief | PHQ-9 > 9 (n = 74) | 17 | 23.0 | 11 | 64.7 |
| Cognitive coping | PHQ-9 > 9 (n = 74) | 28 | 37.8 | 15 | 53.6 |
| Post-traumatic stress disorder | PCL-5 > 32 (n = 35) | 12 | 34.3 | 6 | 50.0 |
| Sleep | ISI >9 (n = 79) | 32 | 40.5 | 16 | 50.0 |
| Worry | GAD-7 > 9 (n = 58) | 18 | 31.0 | 13 | 72.2 |
3.5.2.Relationship to satisfaction and outcomes
The phi correlations and point-biserial correlations (rpb) between number of additional resources accessed, and number of additional resources rated as informative and/or helpful and satisfaction variables revealed no statistically significant associations. Similarly, these variables were not related to any of the outcome variables, with the one exception. Rating more resources as informative and/or helpful was associated with lower WSAS scores (rpb = −0.23, p < .05).
3.5.3.Qualitative feedback
In total, 91 patients provided feedback with most (n = 70) providing feedback at both time-points. All the feedback was combined and analyzed generating 161 comments. Across the 161 comments, 73.3 % (n = 118) of the comments provided no suggestions (e.g. “None that I can think of” 22,442), 16.8 % (n = 27) of comments were about expanding on a topic already in the course (alcohol-related content: n = 14; PTSD resource: n = 7; worry resource: n = 4; grief resource: n = 1; cognitive coping resource: n = 1), and 9.9 % (n = 16) suggested adding a new topic. New topics were generally mentioned only once at mid-treatment or post-treatment (e.g., narcissism, creating habits). The only new topic that was raised by several patients was to offer a resource on relationships (n = 7, 4.3 %, e.g. “Relationship and break up information”22,426).
4.Discussion
In this study, we aimed to evaluate the inclusion of additional resources and diverse patient stories in an existing ICBT alcohol misuse course, called the ACCE. We explored how effective this enhanced course was, how patients would use and rate these enhancements, and how these enhancements would relate to satisfaction and outcomes. Overall, the course was found to result in large effects in reducing weekly alcohol consumption and HDD, along with significant improvements on most secondary outcomes. Most patients (62.8 %) accessed at least one resource with resources on worry, anger, cognitive coping, and grief perceived to be the most helpful. Further, most patients (85.4 %) reported reviewing the patient stories and reported several benefits to reviewing them, such as feeling less alone, and gaining ideas for skill use to improve their wellbeing. Treatment satisfaction was high, and completion of the main lessons was strong, with 68 % of patients completing at least 4 of 6 lessons. However, accessing and rating more resources as informative and/or helpful was not associated with treatment satisfaction or outcomes to a significant degree. In contrast, rating stories more positively was associated with greater confidence in self-managing symptoms, and increased motivation to seek treatment if needed. Reading stories was also associated with larger changes scores on several secondary measures, including anger, PTSD, insomnia, and work and social adjustment.
Direct comparison to past courses is complicated by the differing numbers of modules, but the enhanced intervention generally appears consistent with past research in terms of completion rates. For example, in previous versions, 75.7 % completed at least 5 out of 8 lessons (Sapkota et al., 2024), and in an earlier 12-lesson version of the course (Sundström et al., 2022), patients completed an average of 8.3 lessons (SD = 3.8). In comparison to the broader literature, a review of ICBT for alcohol misuse found that 58–77 % of lessons were completed in guided ICBT courses and 21 % of lessons were completed in a self-guided course (Hadjistavropoulos et al., 2020b).
In terms of outcomes, the results showed that outcomes of the enhanced course were maintained and of a similar magnitude to those found in our past trials in terms of changes in alcohol-related outcomes (Sapkota et al., 2024; Sundström et al., 2022). We found large effect sizes for reductions in total drinks per week as well as decreased HDD, which appear to be as good or better than outcomes reported in a review of therapist-guided ICBT and self-guided ICBT for alcohol misuse (Hadjistavropoulos et al., 2020b). In terms of secondary outcomes, we also found the course reduced symptoms of depression, anxiety, cravings, and risky or hazardous alcohol consumption as we have identified in past studies (Sundström et al., 2022; Sapkota et al., 2024). New to the current study, we observed changes on measures of anger, PTSD, insomnia and work and social adjustment, which we had not previously studied. Addressing comorbid concerns within an ICBT course for alcohol misuse is important, as there are high rates of comorbidity, ranging from 27 to 40 % for depression (Castillo-Carniglia et al., 2019), 20–40 % for anxiety (Castillo-Carniglia et al., 2019), 34–55 % for PTSD (Debell et al., 2014), and 13–41 % for problems with drug use (Esmaeelzadeh et al., 2018; Thompson et al., 2021). Anger is also a common concern among individuals struggling with alcohol misuse (Coccaro et al., 2016), as well as relationship and communication difficulties (Roberts and Linney, 2000). Finally, rates of insomnia symptoms are as high as 74 % among individuals with alcohol use disorder (Brower et al., 2001). These rates are in line with difficulties identified in the current study based on scores on measures at screening (e.g., 65 % had scores suggestive of insomnia, 61 % had clinically elevated depression, 48 % had clinically elevated anxiety, 43 % had relationship discord, 26 % had clinically elevated PTSD, 22 % had clinically elevated anger). We did not observe changes on measures of cannabis use, relationship satisfaction, or motivation. The findings related to cannabis use are not surprising, as only 20 patients scored above the cut-off on the CUDIT-R, with only 10 (50 %) of those patients reviewing the resource. Treatment complexity and burden has recently been discussed as a potential problem when attempting to improve internet interventions (Cross and Alvarez-Jimenez, 2024). Therefore, it is positive to observe large effects remained on primary outcomes, even though we added to the complexity of the intervention by offering additional resources and patient stories. Perhaps the burden of added complexity was offset by other changes we made to the program in terms of condensing alcohol program materials from 8 into 6 lessons.
In terms of additional resources, we found that between 19.8 % to 34.7 % of patients in the full sample accessed each resource. The most accessed resources in the overall sample were the resources on anger (34.7 %), cognitive coping (34.7 %), and sleep (33.9 %). Overall, the resources were rated positively in terms of their informativeness and/or helpfulness. However, four resources were rated as informative and/or helpful by <50 % of patients who accessed them including cannabis use (25.0 %), PTSD (40.0 %), assertiveness and communication (40.6 %), and sleep (46.3 %). These resources should likely be revised to ensure the content is meeting the needs of patients who access the resources.
Ideally, we would expect that patients who scored in the clinical range for a specific concern (e.g., PTSD) would access the corresponding resource, and we found that between 23.0 % (grief) and 50.0 % (cannabis use) of patients did so. Importantly, when used by patients with clinical concerns, the additional resources were often positively evaluated – such as with anger (91 %), worry (72.2 %), grief (64.8 %), and cognitive coping (53.7 %). Overall, there is sufficient use of each resource to make them worthwhile to retain, in that they appear to be used by a significant percentage of the patients (>23 %). Review of the ratings themselves suggest that improving cannabis use resource and assertiveness/communication resource could be beneficial. Of note, qualitative feedback from patients suggested that topics in the course or some of the additional resources could be expanded on (e.g., PTSD, worry) or that a resource on relationships could be added.
Overall, resource use and ratings were not significantly related to treatment satisfaction or outcomes. Given the significant variability in resource use, it is possible that the sample size was not large enough to identify relationships. Additionally, the method used to assess resources (e.g., calculating the number of resources accessed and those marked as informative and or helpful) may not have been sensitive enough to identify associations with satisfaction and outcomes. Of note, rating more additional resources as informative and/or helpful was unexpectedly related to lower improvement in work and social adjustment. A possible explanation is that patients who were struggling more during the course were more likely to access the additional resources. However, this correlation was weak and further research is warranted to understand how individual resources impact outcomes.
In terms of the stories, there appears to be quite high engagement with them (85.4 %) and most patients found them worthwhile (89.0 %). Patient ratings of the stories suggest that they fulfilled the five purposes of patient stories in online healthcare including: relatability, providing comfort, increased knowledge, modeling desired behaviours, and increased motivation to use the skills (Shaffer and Zikmund-Fisher, 2013). Patients who rated the stories more positively also had increased confidence in managing their symptoms, as well as increased motivation to seek treatment in the future if needed. Further, there were small, but significant positive relationships between reading stories and improvements on measures of PTSD, anger, insomnia, and work and social adjustment. These relationships though were weak and because the data is correlational, it is difficult to know the meaning of the relationships. Qualitative feedback from patients was limited, with most patients not offering a suggested improvement. While some ideas were generated for improving stories, overall, most patients had no feedback and feedback that was obtained was highly diverse and at times contradictory (e.g., add, reduce or remove stories) suggesting that significant changes to stories are not required at this time.
4.1.Limitations and future directions
This study was primarily observational and correlational in nature, which limits the conclusions that can be drawn. We examined the association between the enhanced course content (i.e., additional resources and patient stories) and satisfaction and outcomes, but the design of the trial did not allow for a causal understanding of each of the component's contributions. Further, the sample size may have not been large enough to reflect the strength of the relationships between additional resources, stories and satisfaction and outcomes. Other limitations relate to the sample characteristics (e.g., limited racial diversity), measures that may not have been sensitive to change (e.g., the motivation measure, which demonstrated high levels of motivation at all time points), and the quality of the questions used to evaluate the additional resources (e.g., asking patients to identify those that were informative and or helpful) and stories (e.g., asked about all stories as a whole at 4 and 8 weeks).
Future studies on additional resources could vary the way that resources are selected (e.g., comparing outcomes when resources are self-selected versus therapist-recommended), the timing of the additional resources (e.g., during the treatment versus targeted work with resources after the core lessons are completed), or the tasks that are associated with the resources (e.g., including specific homework as part of the resources versus the resource in its standard form). Additional research could also have clients rate resources accessed using Likert scales and include open-ended questions or interviews to gather more feedback about how to improve the less favorably rated resources (e.g., assertiveness and cannabis use). Future research could improve the relatability of personal stories by using large language models (LLMs; Bhattacharjee et al., 2024). LLMs have the potential to create personalized or tailored stories based on patients' descriptions of their symptoms and situations. An interesting direction for research would be to randomize patients to receive patient stories that are generated with LLMs compared to “stock” stories that are identical for all patients. Future studies should also aim to include a more diverse sample (e.g., greater ethnic diversity, variety in educational backgrounds) to explore whether the findings are generalizable in diverse populations.
5.Conclusions
Overall, the findings of the current study suggest that adding relevant resources and stories to ICBT for alcohol use seems acceptable and worthwhile. A significant number of patients read the stories and found them beneficial. Moreover, a significant number accessed and found resources informative and/or helpful, which suggests it is likely valuable to retain these resources for those in need. There are multiple future research directions that could be taken to better understand how stories and resources relate to treatment satisfaction and outcomes, including incorporating Likert scales and open-ended feedback to refine less favorably rated resources, using LLMs to generate personalized stories, and ensuring a more diverse sample to improve generalizability. Importantly, despite adding more complexity in terms of stories and additional resources and potential burden to the ICBT intervention, good clinical outcomes and high levels of treatment completion and satisfaction were still observed.
Funding
This research was supported by funding from the Saskatchewan Centre for Patient-Oriented Research and the Saskatchewan Health Research Foundation. The Online Therapy Unit is funded by the Saskatchewan Ministry of Health to provide screening and treatment. Funders had no involvement in the study design, collection, analysis, or interpretation of the data.
Declaration of competing interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgements
The authors would like to thank members of the Patient-Oriented Research Steering Committee who contributed to all aspects of the research grant application, to revision of materials, to advertising, and to interpreting results. The authors also wish to acknowledge the patients, screeners, therapists, research staff, students, and web developers associated with the Online Therapy Unit at the University of Regina. We would specifically like to acknowledge Andrew Wilhelms for his assistance with data cleaning.
Footnotes
Footnote Group
References
Untitled section
References
- Adamson S.J., Kay-Lambkin F.J., Baker A.L., Lewin T.J., Thornton L., Kelly B.J., Sellman J.D. An improved brief measure of cannabis misuse: the Cannabis use disorders identification test – revised (CUDIT-R) Drug Alcohol Depend. 2010;110:137–143. doi: 10.1016/j.drugalcdep.2010.02.017.
- Andersson G. Internet-delivered psychological treatments. Annu. Rev. Clin. Psychol. 2016;12:157–179. doi: 10.1146/annurev-clinpsy-021815-093006.
- Baumgartner C., Schaub M.P., Wenger A., Malisching D., Augsburger M., Lehr D., Blankers M., Ebert D.D., Haug S. “Take Care of You” – efficacy of integrated, minimal-guidance, internet-based self-hep for reducing co-occurring alcohol misuse and depression symptoms in adults: results of a three-arm randomized controlled trial. Drug Alcohol Depend. 2021;225 doi: 10.1016/j.drugalcdep.2021.108806.
- Berman A.H., Bergman H., Palmstierna T., Schlyter F. Drug use disorders identification test (DUDIT) APA PsycTests. 2002 doi: 10.1037/t02890-000.
- Bhattacharjee A., Xu S.Y., Rao P., Zeng Y., Meyerhoff J., Ahmed S.I., Mohr D.C., Liut M., Mariakakis A., Kornfield R., Williams J.J. “It explains what I am currently going through perfectly to a tee”: understanding user perceptions on LLM-enhanced narrative interventions. ACM. 2024 doi: 10.48550/arXiv.2409.16732.
- Blevins C.A., Weathers F.W., Davis M.T., Witte T.K., Domino J.L. The posttraumatic stress disorder checklist for DSM-5 (PCL-5): development and initial psychometric evaluation. J. Trauma. Stress. 2015;28(6):489–498. doi: 10.1002/jts.22059.
- Brett J., Staniszewska S., Mockford C., Herron-Marx S., Hughes J., Tysall C., Suleman R. Mapping the impact of patient and public involvement on health and social care research: A systematic review. Health Expect. 2014;17:637–650. doi: 10.1111/j.1369-7625.2012.00795.
- Brower K.J., Aldrich M.S., Robinson E.A., Zucker R.A., Greden J.F. Insomnia, self-medication, and relapse to alcoholism. Am. J. Psychiatry. 2001;158:399–404. doi: 10.1111/j.1530-0277.2008.00706.x.
- Castillo-Carniglia A., Keyes K.M., Hasin D.S., Cerdá M. Psychiatric comorbidities in alcohol use disorder. Lancet Psychiatry. 2019;6(12):1068–1080. doi: 10.1016/S2215-0366(19)30222-6.
- Coccaro E.F., Fridberg D.J., Fanning J.R., Grant J.E., King A.C., Lee R. Substance use disorders: relationship with intermittent explosive disorder and with aggression, anger, and impulsivity. J. Psychiatr. Res. 2016;81:127–132. doi: 10.1016/j.jpsychires.2016.06.011.
- Cohen J. 2nd ed. Erlbaum; 1988. Statistical power analysis for the behavioral sciences.
- Cross S.P., Alvarez-Jimenez M. The digital cumulative complexity model: A framework for improving engagement in digital mental health interventions. Front. Psych. 2024;15 doi: 10.3389/fpsyt.2024.1382726.
- Debell F., Fear N.T., Head M., Batt-Rawden S., Greenberg N., Wessely S., Goodwin L. A systematic review of the comorbidity between PTSD and alcohol misuse. Soc. Psychiatry Psychiatr. Epidemiol. 2014;49(9):1401–1425. doi: 10.1007/s00127-014-0855-7.
- Degenhardt L., Bharat C., Chiu W.T., Harris M.G., Kazdin A.E., Vigo D.V., Sampson N.A., Alonso J., Andrade L.H., Bruffaerts R., Bunting B., Cardoso G., de Girolamo G., Florescu S., Giureje O., Haro J.M., Hu C., Karam E.G., Kovess-Masfety V.…Wojtyniak B. Perceived helpfulness of treatment for alcohol use disorders: findings from the World Health Surveys. Drug Alcohol Depend. 2021;229 doi: 10.1016/j.drugalcdep.2021.109158.
- Devilly G.J., Borkovec T.D. Psychometric properties of the credibility/expectancy questionnaire. J. Behav. Ther. Exp. Psychiatry. 2000;31(2):73–86. doi: 10.1016/S0005-7916(00)00012-4.
- Enders C.K. Guilford Publications; 2022. Applied missing data analysis.
- Esmaeelzadeh S., Morarors J., Thorpe L., Bird Y. Examining the association and directionality between mental health disorders and substance use among adolescents and young adults in the US and Canada – A systematic review and meta-analysis. J. Clin. Med. 2018;7(12):543. doi: 10.3390/jcm7120543. doi:103390/jcm7120543.
- Flannery B.A., Volpicelli J.R., Pettinati H.M. Psychometric properties of the Penn alcohol craving scale. Alcohol Clinical & Experimental Research. 1999;23(8):1289–1995. doi: 10.1111/j.1530-0277.1999.tb04349.x.
- Forbes D., Alkemade N., Mitchell D., Elhai J.D., Mchugh T., Bates G., Lewis V. Utility of the dimension of anger Reactions-5 (DAR-5) scale as a brief anger measure. Depress. Anxiety. 2014;31(2):166–173. doi: 10.1002/da.22148.
- Frohlich J.R., Rapinda K.K., Schaub M.P., Wenger A., Baumgartner C., Johnson E.A., O’Connor R.M., Vincent N., Blankers M., Ebert D.D., Hadjistavropoulos H.D., Mackenzie C.S., Wardell J.D., Augsburger M., Goldberg J.O., Keough M.T. Efficacy of a minimally guided internet treatment for alcohol misuse and emotional problems in young adults: results of a randomized controlled trial. Addict. Behav. Rep. 2021;14 doi: 10.1016/j.abrep.2021.100390.
- Funk J.L., Rogge R.D. Testing the ruler with item response theory: increasing precision of measurement for relationship satisfaction with the couples satisfaction index. J. Fam. Psychol. 2007;21:572–583. doi: 10.13140/RG.2.1.4198.3129.
- GBD 2020 Alcohol Collaborators Population-level risks of alcohol consumption by amount, geography, age, sex, and year: A systematic analysis for the Global Burden of Disease 2020. Lancet. 2020;400(10347):185–235. doi: 10.1016/S0140-6736(22)00847-9.
- Graham J.W., Olchowski A.E., Gilreath T.D. How many imputations are really needed? Some practical clarifications of multiple imputation theory. Prevention Science. 2007;8:206–213. doi: 10.1007/s11121-007-0070-9.
- Hadjistavropoulos H.D., Alberts N., Nugent M., Marchildon G. Improving access to psychological services through therapist-assisted Internet cognitive behaviour therapy. Canadian Psychology. 2014;55(4):303–311. doi: 10.1037/a0037716.
- Hadjistavropoulos H.D., Gullickson K.M., Adrian-Taylor S., Wilhelms A., Sundström C., Nugent M. Stakeholder perceptions of internet-delivered cognitive behaviour therapy as a treatment option for alcohol misuse: A qualitative analysis. JMIR Mental Health. 2020;7(3) doi: 10.2196/14698.
- Hadjistavropoulos H.D., Mehta S., Wilhelms A., Keough M.T., Sundström C. A systematic review of internet-delivered cognitive behavior therapy for alcohol misuse: study characteristics, program content and outcomes. Cogn. Behav. Ther. 2020;49(4):327–346. doi: 10.1080/16506073.2019.1663258.
- Hadjistavropoulos H.D., Peynenburg V., Thiessen D.L., Nugent M., Adlam K., Owens K.M.B., Karin E., Dear B.F., Titov N. A pragmatic factorial randomized controlled trial of transdiagnostic internet-delivered cognitive behavioural therapy: exploring benefits of homework reflection questionnaires and twice-weekly therapist support. Internet Interv. 2020;22 doi: 10.1016/j.invent.2020.100357.
- Hadjistavropoulos H.D., Chadwick C., Beck C.D., Edmonds M., Sundstrom C., Edwards W., Ouellette D., Waldrop J., Adlam K., Bourgeault L., Nugent M. Improving internet-delivered cognitive behaviour therapy for alcohol misuse: patient perspectives following program completion. Internet Interv. 2021;26 doi: 10.1016/j.invent.2021.100474.
- Hadjistavropoulos H.D., Peynenburg V., Thiessen D., Nugent M., Karin E., Staples L., Dear B., Titov N. Utilization, patient characteristics and longitudinal improvements among patients from a provincially funded transdiagnostic internet-delivered cognitive behavioural therapy program: observational study of trends over six years. Can. J. Psychiatry. 2021;67(3):1–15. doi: 10.1177/07067437211006873.
- Hadjistavropoulos H.D., Hill T.G., Philipp-Muller A.E., Dear B., Titov N. Perceptions of client stories in internet-delivered cognitive behaviour therapy: A mixed-methods evaluation. Internet Interv. 2023;34 doi: 10.1016/j.invent.2023.100692.
- Hadjistavropoulos H.D., Peynenburg V., Sapkota R.P., Titov N., Dear B.F. Evaluation of additional resources used in therapist-assisted transdiagnostic internet-delivered cognitive behaviour therapy. Internet Interv. 2024;37 doi: 10.1016/j.invent.2024.100758.
- Hsieh H.F., Shannon S.E. Three approaches to qualitative content analysis. Qual. Health Res. 2005;15(9):1277–1288. doi: 10.1016/j.invent.2021.100450.
- Kroenke K., Spitzer R.L., Williams J.B. The PHQ-9: validity of a brief depression severity measure. J. Gen. Intern. Med. 2001;16(9):606–613. doi: 10.1046/j.1525-1497.2001.016009606.x.
- Little R.J., Rubin D.B. John Wiley and Sons; 2020. Statistical Analysis with Missing Data.
- Mekonen T., Chan G.C.K., Connor J., Hall W., Hides L., Leung J. Treatment rates for alcohol use disorders: A systematic review and meta-analysis. Addiction. 2020;116(10):2617–2634. doi: 10.1111/add.15357.
- Morin C.M., Belleville G., Belanger L., Ivers H. The insomnia severity index: psychometric indicators to detect insomnia cases and evaluate treatment response. Sleep. 2011;34(5):601–608. doi: 10.1093/sleep/34.5.601.
- Mundt J.C., Marks I.M., Shear M.K., Griest J.H. The work and social adjustment scale: A simple measure of impairment in functioning. Br. J. Psychiatry. 2002;180:461–464. doi: 10.1192/bjp.180.5.461.
- Peugh J.L., Enders C.K. Using the SPSS mixed procedure to fit cross-sectional and longitudinal multilevel models. Educ. Psychol. Meas. 2005;65(5):717–741.
- Preference Collaborative Review Group Patients’ preferences within randomised trials: systematic review and patient level meta-analysis. BMJ. 2008;337 doi: 10.1136/bmj.a1864.
- Price M., Szafranski D.D., van Stolk-Cooke K., Gros D.F. Investigation of abbreviated 4 and 8 item versions of the PTSD Checklist 5. Psychiatry Res. 2016;239:124–130. doi: 10.1016/j.psychres.2016.03.014.
- Roberts L.J., Linney K.D. In: The Psychology of Couples and Illness: Theory, Research, & Practice. Schmaling K.B., Sher T.G., editors. American Psychological Association; 2000. Alcohol problems and couples: Drinking in an intimate relational context; pp. 269–310.
- Sapkota R.P., Lozinski T., Wilhems A., Nugent M., Schaub M.P., Keough M.T., Sundstrom C., Hadjistavropoulos H.D. Internet-delivered therapy for alcohol misuse: engagement, satisfaction, and outcomes when patients select their preference for therapist- or self-guided treatment. Addict. Sci. Clin. Pract. 2024;19(1):30. doi: 10.1186/s13722-024-00456-8.
- Saunders J.B., Aasland O.G., Babor T.F., De La Fuente J.R., Grant M. Development of the alcohol use disorders identification test (AUDIT): WHO collaborative project on early detection of persons with harmful consumption-II. Addiction. 1993;88(6):791–804. doi: 10.1111/j.1360-0443.1993.tb02093.x.
- Schaub M.P., Blankers M., Lehr D., Boss L., Riper H., Dekker J., Goudriaan A.E., Maier L.J., Haug S., Amann M., Dey M., Wenger A., Ebert D.D. Efficacy of an internet-based self-help intervention to reduce co-occurring alcohol misuse and depression symptoms in adults: study protocol of a three-arm randomized controlled trial. BMJ Open. 2016;6 doi: 10.1136/bmjopen-2016-011457.
- Schuler M.S., Puttaiah S., Mojtabai R., Crum R.M. Perceived barriers to treatment for alcohol problems: A latent class analysis. Psychiatr. Serv. 2015;66(11):1221–1228. doi: 10.1176/appi.ps.201400160.
- Shaffer V.A., Zikmund-Fisher B.J. All stories are not alike: A purpose-, content-, and valence-based taxonomy of patient narratives in decision aids. Med. Decis. Making. 2013;33(1):4–13. doi: 10.1177/0272989X12463266.
- Skivington K., Matthews L., Simpson S.A., Baird J., Boyd K.A., French D.P., Petticrew M., White M. A new framework for developing and evaluating complex interventions: update of Medical Research Council Guidance. BMJ. 2021;374 doi: 10.1136/bmj.n2061.
- Sobell L., Sobell M. In: Measuring Alcohol Consumption: Psychosocial and Biological Methods. Allen J., Litten R., editors. Humana Press; 1992. Timeline follow-back: A technique for assessing self-reported ethanol consumption; pp. 41–72.
- Spitzer R.L., Kroenke K., Williams J.B.W., Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch. Intern. Med. 2006;166(10):1092–1097. doi: 10.1001/archinte.166.10.1092.
- Sundström C., Schell C., Wardell J., Godinho A., Cunningham J. Can brief email guidance enhance the effects of an internet intervention for people with problematic alcohol use? A randomized controlled trial. Subst. Use Misuse. 2020;55(12):2011–2019. doi: 10.1080/10826084.2020.1788087.
- Sundström C., Peynenburg V., Chadwick C., Thiessen D., Wilhelms A., Nugent M., Keough M.T., Schaub M.P., Hadjistavropolous H.D. Optimizing internet-delivered cognitive behaviour therapy for alcohol misuse – A randomized factorial trial exmaining effects of a pre-treatment assessment interview and guidance. Addict. Sci. Clin. Pract. 2022;17:37. doi: 10.1186/s13722-022-00319-0.
- Thompson K., Holley M., Sturgess C., Leadbeater B. Co-use of alcohol and cannabis: longitudinal associations with mental health outcomes in young adulthood. Int. J. Environ. Res. Public Health. 2021;18(7):3652. doi: 10.3390/ijerph18073652.
- Twisk J., de Boer M., de Vente W., Heymans M. Multiple imputation of missing values was not necessary before performing a longitudinal mixed-model analysis. J. Clin. Epidemiol. 2013;66(9):1022–1028. doi: 10.1016/j.jclinepi.2013.03.017.
- Weathers F.W., Blake D.D., Schnurr P.P., Kaloupek D.G., Marx B.P., Keane T.M. The Life Events Checklist for DSM-5 (LEC-5) 2013. www.ptsd.va.gov
- Zuromski K.L., Ustun B., Hwang I., Keane T.M., Marx B.P., Stein M.B., Ursano R.J., Kessler R.C. 2019. Developing an Optimal Short-Form of the PTSD Checklist for DSM-5 (PCL-5)