Digital Interventions for Recreational Cannabis Use Among Young Adults: Systematic Review, Meta-Analysis, and Behavior Change Technique Analysis of Randomized Controlled Studies
Faculty of Nursing, Université de Montréal, Montreal, QC, Canada
Research Centre of the Centre Hospitalier de l’Université de Montréal, Montreal, QC, Canada
Research Chair in Innovative Nursing Practices, Montreal, QC, Canada
Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael’s Hospital, Toronto, ON, Canada
Department of Nursing, Université du Québec en Outaouais, Saint-Jérôme, QC, Canada
Women's College Hospital Institute for Health System Solutions and Virtual Care, Women's College Hospital, Toronto, ON, Canada
Ingram School of Nursing, Faculty of Medicine and Health Sciences, McGill University, Montreal, QC, Canada
Centre for Clinical Epidemiology, Lady Davis Institute for Medical Research, Sir Mortimer B. Davis Jewish General Hospital, Montreal, QC, Canada
Kirby Institute, University of New South Wales, Sydney, Australia
Department of Psychiatry and Addictology, Faculty of Medicine, Université de Montréal, Montreal, QC, Canada
Abstract
Background
The high prevalence of cannabis use among young adults poses substantial global health concerns due to the associated acute and long-term health and psychosocial risks. Digital modalities, including websites, digital platforms, and mobile apps, have emerged as promising tools to enhance the accessibility and availability of evidence-based interventions for young adults for cannabis use. However, existing reviews do not consider young adults specifically, combine cannabis-related outcomes with those of many other substances in their meta-analytical results, and do not solely target interventions for cannabis use.
Objective
We aimed to evaluate the effectiveness and active ingredients of digital interventions designed specifically for cannabis use among young adults living in the community.
Methods
We conducted a systematic search of 7 databases for empirical studies published between database inception and February 13, 2023, assessing the following outcomes: cannabis use (frequency, quantity, or both) and cannabis-related negative consequences. The reference lists of included studies were consulted, and forward citation searching was also conducted. We included randomized studies assessing web- or mobile-based interventions that included a comparator or control group. Studies were excluded if they targeted other substance use (eg, alcohol), did not report cannabis use separately as an outcome, did not include young adults (aged 16-35 y), had unpublished data, were delivered via teleconference through mobile phones and computers or in a hospital-based setting, or involved people with mental health disorders or substance use disorders or dependence. Data were independently extracted by 2 reviewers using a pilot-tested extraction form. Authors were contacted to clarify study details and obtain additional data. The characteristics of the included studies, study participants, digital interventions, and their comparators were summarized. Meta-analysis results were combined using a random-effects model and pooled as standardized mean differences.
Results
Of 6606 unique records, 19 (0.29%) were included (n=6710 participants). Half (9/19, 47%) of these articles reported an intervention effect on cannabis use frequency. The digital interventions included in the review were mostly web-based. A total of 184 behavior change techniques were identified across the interventions (range 5-19), and feedback on behavior was the most frequently used (17/19, 89%). Digital interventions for young adults reduced cannabis use frequency at the 3-month follow-up compared to control conditions (including passive and active controls) by −6.79 days of use in the previous month (95% CI −9.59 to −4.00; P<.001).
Conclusions
Our results indicate the potential of digital interventions to reduce cannabis use in young adults but raise important questions about what optimal exposure dose could be more effective, both in terms of intervention duration and frequency. Further high-quality research is still needed to investigate the effects of digital interventions on cannabis use among young adults.
Trial Registration
PROSPERO CRD42020196959; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=196959
Untitled section
Keywords: cannabis, young adults, digital intervention, systematic review, meta-analysis, behavior change technique analysis, mobile phone
Article notes
Untitled section
Received 2023 Nov 30; Revision requested 2024 Jan 2; Revised 2024 Jan 9; Accepted 2024 Mar 8; Collection date 2024.
Introduction
Cannabis Use Among Young Adults Is Recognized as a Public Health Concern
Young adulthood (typically the ages of 18-30 y) is a critical developmental stage characterized by a peak prevalence of substance use [1,2]. Worldwide, cannabis is a substance frequently used for nonmedical purposes due in part to its high availability in some regions and enhanced product variety and potency [3,4]. The prevalence of cannabis use (CU) among young adults is high [5,6], and its rates have risen in recent decades [7]. In North America and Oceania, the estimated past-year prevalence of CU is ≥25% among young adults [8,9].
While the vast majority of cannabis users do not experience severe problems from their use [4], the high prevalence of CU among young adults poses substantial global health concerns due to the associated acute and long-term health and psychosocial risks [10,11]. These include impairment of cognitive function, memory, and psychomotor skills during acute intoxication; increased engagement in behaviors with a potential for injury and fatality (eg, driving under the influence); socioeconomic problems; and diminished social functioning [4,12-14]. Importantly, an extensive body of literature reveals that subgroups engaging in higher-risk use, such as intensive or repeated use, are more prone to severe and chronic consequences, including physical ailments (eg, respiratory illness and reproductive dysfunction), mental health disorders (eg, psychosis, depression, and suicidal ideation or attempts), and the potential development of CU disorder [4,15-17].
Interventions to Reduce Public Health Impact of Young Adult CU
Given the increased prevalence of lifetime and daily CU among young adults and the potential negative impact of higher-risk CU, various prevention and intervention programs have been implemented to help users reduce or cease their CU. These programs primarily target young adults regardless of their CU status [2,18]. In this context, many health care organizations and international expert panels have developed evidence-based lower-risk CU guidelines to promote safer CU and intervention options to help reduce risks of adverse health outcomes from nonmedical CU [4,16,17,19]. Lower-risk guidance-oriented interventions for CU are based on concepts of health promotion [20-22] and health behavior change [23-26] and on other similar harm reduction interventions implemented in other areas of population health (eg, lower-risk drinking guidelines, supervised consumption sites and services, and sexual health) [27,28]. These interventions primarily aim to raise awareness of negative mental, physical, and social cannabis-related consequences to modify individual-level behavior-related risk factors.
Meta-analyses have shown that face-to-face prevention and treatment interventions are generally effective in reducing CU in young adults [18,29-32]. However, as the proportion of professional help seeking for CU concerns among young adults remains low (approximately 15%) [33,34], alternative strategies that consider the limited capacities and access-related barriers of traditional face-to-face prevention and treatment facilities are needed. Digital interventions, including websites, digital platforms, and mobile apps, have emerged as promising tools to enhance the accessibility and availability of evidence-based programs for young adult cannabis users. These interventions address barriers such as long-distance travel, concerns about confidentiality, stigma associated with seeking treatment, and the cost of traditional treatments [35-37]. By overcoming these barriers, digital interventions have the potential to have a stronger public health impact [18,38].
State of Knowledge of Digital Interventions for CU and Young Adults
The literature regarding digital interventions for substance use has grown rapidly in the past decade, as evidenced by several systematic reviews and meta-analyses of randomized controlled trial (RCT) studies on the efficacy or effectiveness of these interventions in preventing or reducing harmful substance use [2,39-41]. However, these reviews do not focus on young adults specifically. In addition, they combine CU-related outcomes with those of many other substances in their meta-analytical results. Finally, they do not target CU interventions exclusively.
In total, 4 systematic reviews and meta-analyses of digital interventions for CU among young people have reported mixed results [42-45]. In their systematic review (10 studies of 5 prevention and 5 treatment interventions up to 2012), Tait et al [44] concluded that digital interventions effectively reduced CU among adolescents and adults at the posttreatment time point. Olmos et al [43] reached a similar conclusion in their meta-analysis of 9 RCT studies (2 prevention and 7 treatment interventions). In their review, Hoch et al [42] reported evidence of small effects at the 3-month follow-up based on 4 RCTs of brief motivational interventions and cognitive behavioral therapy (CBT) delivered on the web. In another systematic review and meta-analysis, Beneria et al [45] found that web-based CU interventions did not significantly reduce consumption. However, these authors indicated that the programs tested varied significantly across the studies considered and that statistical heterogeneity was attributable to the inclusion of studies of programs targeting more than one substance (eg, alcohol and cannabis) and both adolescents and young adults. Beneria et al [45] recommend that future work “establish the effectiveness of the newer generation of interventions as well as the key ingredients” of effective digital interventions addressing CU by young people. This is of particular importance because behavior change interventions tend to be complex as they consist of multiple interactive components [46].
Behavior change interventions refer to “coordinated sets of activities designed to change specified behavior patterns” [47]. Their interacting active ingredients can be conceptualized as behavior change techniques (BCTs) [48]. BCTs are specific and irreducible. Each BCT has its own individual label and definition, which can be used when designing and reporting complex interventions and as a nomenclature system when coding interventions for their content [47]. The Behavior Change Technique Taxonomy version 1 (BCTTv1) [48,49] was developed to provide a shared, standardized terminology for characterizing complex behavior change interventions and their active ingredients. Several systematic reviews with meta-regressions that used the BCTTv1 have found interventions with certain BCTs to be more effective than those without [50-53]. A better understanding of the BCTs used in digital interventions for young adult cannabis users would help not only to establish the key ingredients of such interventions but also develop and evaluate effective interventions.
In the absence of any systematic review of the effectiveness and active ingredients of digital interventions designed specifically for CU among community-living young adults, we set out to achieve the following:
- conduct a comprehensive review of digital interventions for preventing, reducing, or ceasing CU among community-living young adults,
- describe the active ingredients (ie, BCTs) in these interventions from the perspective of behavior change science, and
- analyze the effectiveness of these interventions on CU outcomes.
Methods
Protocol Registration
We followed the Cochrane Handbook for Systematic Reviews of Interventions [54] in designing this systematic review and meta-analysis and the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2020 guidelines in reporting our findings (see Multimedia Appendix 1 [55] for the complete PRISMA checklist). This review was registered in PROSPERO (CRD42020196959).
Search Strategy
Overview
The search strategy was designed by a health information specialist together with the research team and peer reviewed by another senior information specialist before execution using Peer Review of Electronic Search Strategies for systematic reviews [56]. The search strategy revolved around three concepts:
- CU (eg, “cannabis,” “marijuana,” and “hashish”)
- Digital interventions (eg, “telehealth,” “website,” “mobile applications,” and “computer”)
- Young adults (eg, “emerging adults” and “students”)
The strategy was initially implemented on March 18, 2020, and again on October 13, 2021, and February 13, 2023. The full, detailed search strategies for each database are presented in Multimedia Appendix 2.
Information Sources
We searched 7 electronic databases of published literature: CINAHL Complete, Cochrane Database of Systematic Reviews, Cochrane Central Register of Controlled Trials, Embase, MEDLINE, PubMed, and PsycINFO. No publication date filters or language restrictions were applied. A combination of free-text keywords and Medical Subject Headings was tailored to the conventions of each database for optimal electronic searching. The research team also manually screened the reference lists of the included articles and the bibliographies of existing systematic reviews [18,31,42-45] to identify additional relevant studies (snowballing). Finally, a forward citation tracking procedure (ie, searching for articles that cited the included studies) was carried out in Google Scholar.
Inclusion Criteria
The population, intervention, comparison, outcome, and study design process is presented in Multimedia Appendix 3. The inclusion criteria were as follows: (1) original research articles published in peer-reviewed journals; (2) use of an experimental study design (eg, RCT, cluster RCT, or pilot RCT); (3) studies evaluating the effectiveness (or efficacy) of digital interventions designed specifically to prevent, reduce, or cease CU as well as promote CU self-management or address cannabis-related harm and having CU as an outcome measure; (4) studies targeting young adults, including active and nonactive cannabis users; (5) cannabis users and nonusers not under substance use treatment used as controls in comparator, waitlist, or delayed-treatment groups offered another type of intervention (eg, pharmacotherapy or psychosocial) different from the one being investigated or participants assessed only for CU; and (6) quantitative CU outcomes (frequency and quantity) or cannabis abstinence. Given the availability of numerous CU screening and assessment tools with adequate psychometric properties and the absence of a gold standard in this regard [57], any instrument capturing aspects of CU was considered. CU outcome measures could be subjective (eg, self-reported number of CU days or joints in the previous 3 months) or objective (eg, drug screening test). CU had to be measured before the intervention (baseline) and at least once after.
Digital CU interventions were defined as web- or mobile-based interventions that included one or more activities (eg, self-directed or interactive psychoeducation or therapy, personalized feedback, peer-to-peer contact, and patient-to-expert communication) aimed at changing CU [58]. Mobile-based interventions were defined as interventions delivered via mobile phone through SMS text message, multimedia messaging service (ie, SMS text messages that include multimedia content, such as pictures, videos, or emojis), or mobile apps, whereas web-based interventions (eg, websites and digital platforms) were defined as interventions designed to be accessed on the web (ie, the internet), mainly via computers. Interventions could include self-directed and web-based interventions with human support. We defined young adults as aged 16 to 35 years and included students and nonstudents. While young adulthood is typically defined as covering the ages of 18 to 30 years [59], we broadened the range given that the age of majority and legal age to purchase cannabis differs across countries and jurisdictions. This was also in line with the age range targeted by several digital CU interventions (college or university students or emerging adults aged 15-24 years) [31,45]. Given the language expertise of the research team members and the available resources, only English- and French-language articles were retained.
Exclusion Criteria
Knowledge synthesis articles, study protocols, and discussion papers or editorials were excluded, as were articles with cross-sectional, cohort, case study or report, pretest-posttest, quasi-experimental, or qualitative designs. Mixed methods designs were included only if the quantitative component was an RCT. We excluded studies if (1) use of substances other than cannabis (eg, alcohol, opioids, or stimulants) was the focus of the digital intervention (though studies that included polysubstance users were retained if CU was assessed and reported separately); (2) CU was not reported separately as an outcome or only attitudes or beliefs regarding, knowledge of, intention to reduce, or readiness or motivation to change CU was measured; and (3) the data reported were unpublished (eg, conferences and dissertations). Studies of traditional face-to-face therapy delivered via teleconference on mobile phones and computers or in a hospital-based setting and informational campaigns (eg, web-based poster presentations or pamphlets) were excluded as well. Studies with samples with a maximum age of <15 years and a minimum age of >35 years were also excluded. Finally, we excluded studies that focused exclusively on people with a mental health disorder or substance use disorder or dependence or on adolescents owing to the particular health care needs of these populations, which may differ from those of young adults [1].
Data Collection
Selection of Studies
Duplicates were removed from the literature search results in EndNote (version X9.3.3; Clarivate Analytics) using the Bramer method for deduplication of database search results for systematic reviews [60]. The remaining records were uploaded to Covidence (Veritas Health Innovation), a web-based systematic review management system. A reviewer guide was developed that included screening questions and a detailed description of each inclusion and exclusion criterion based on PICO (population, intervention, comparator, and outcome), and a calibration exercise was performed before each stage of the selection process to maximize consistency between reviewers. Titles and abstracts of studies flagged for possible inclusion were screened first by 2 independent reviewers (GC, BV, PA, and GR; 2 per article) against the eligibility criteria (stage 1). Articles deemed eligible for full-text review were then retrieved and screened for inclusion (stage 2). Full texts were assessed in detail against the eligibility criteria again by 2 reviewers independently. Disagreements between reviewers were resolved through consensus or by consulting a third reviewer.
Data Extraction Process
In total, 2 reviewers (GC, BV, PA, GR, and GF; 2 per article) independently extracted relevant data (or informal evidence) using a data extraction form developed specifically for this review and integrated into Covidence. The form was pilot-tested on 2 randomly selected studies and refined accordingly. Data pertaining to the following domains were extracted from the included studies: (1) Study characteristics included information on the first and corresponding authors, publication year, country of origin, aims and hypotheses, study period, design (including details on randomization and blinding), follow-up times, data collection methods, and types of statistical analysis. (2) Participant characteristics included study target population, participant inclusion and exclusion criteria, sex or gender, mean age, and sample sizes at each data collection time point. (3) Intervention characteristics, for which the research team developed a matrix inspired by the template for intervention description and replication 12-item checklist [61] to extract informal evidence (ie, intervention descriptions) from the included studies under the headings name of intervention, purpose, underpinning theory of design elements, treatment approach, type of technology (ie, web or mobile) and software used, delivery format (ie, self-directed, human involvement, or both), provider characteristics (if applicable), intervention duration (ie, length of treatment and number of sessions or modules), material and procedures (ie, tools or activities offered, resources provided, and psychoeducational content), tailoring, and unplanned modifications. (4) Comparator characteristics were details of the control or comparison group or groups, including nature (passive vs active), number of groups or clusters (if applicable), type and length of the intervention (if applicable), and number of participants at each data collection time point. (5) Outcome variables, including the primary outcome variable examined in this systematic review, that is, the mean difference in CU frequency before and after the intervention and between the experimental and control or comparison groups. When possible, we examined continuous variables, including CU frequency means and SDs at the baseline and follow-up time points, and standardized regression coefficients (ie, β coefficients and associated 95% CIs). The secondary outcomes examined included other CU outcome variables (eg, quantity of cannabis used and abstinence) and cannabis-related negative consequences (or problems). Details on outcome variables (ie, definition, data time points, and missing data) and measurements (ie, instruments, measurement units, and scales) were also extracted.
In addition, data on user engagement and use of the digital intervention and study attrition rates (ie, dropouts and loss to follow-up) were extracted. When articles had missing data, we contacted the corresponding authors via email (2 attempts were made over a 2-month period) to obtain missing information. Disagreements over the extracted data were limited and resolved through discussion.
Data Synthesis Methods
Descriptive Synthesis
The characteristics of the included studies, study participants, interventions, and comparators were summarized in narrative and table formats. The template for intervention description and replication 12-item checklist [61] was used to summarize and organize intervention characteristics and assess to what extent the interventions were appropriately described in the included articles. As not all studies had usable data for meta-analysis purposes and because of heterogeneity, we summarized the main findings (ie, intervention effects) of the included studies in narrative and table formats for each outcome of interest in this review.
BCT Coding
The BCTs used in the digital interventions were identified from the descriptions of the interventions (ie, experimental groups) provided in the articles as well as any supplementary material and previously published research protocols. A BCT was defined as “an observable, replicable, and irreducible component of an intervention designed to alter or redirect causal processes that regulate behavior” [48]. The target behavior in this review was the cessation or reduction of CU by young adults. BCTs were identified and coded using the BCTTv1 [48,49], a taxonomy of 93 BCTs organized into 16 hierarchical thematic clusters or categories. Applying the BCTTv1 in a systematic review allows for the comparison and synthesis of evidence across studies in a structured manner. This analysis allows for the identification of the explicit mechanisms underlying the reported behavior change induced by interventions, successful or not, and, thus, avoids making implicit assumptions about what works [62].
BCT coding was performed by 2 reviewers independently—BV coded all studies, and GC and GF coded a subset of the studies. All reviewers completed web-based training on the BCTTv1, and GF is an experienced implementation scientist who had used the BCTTv1 in prior work [63-65]. The descriptions of the interventions in the articles were read line by line and analyzed for the clear presence of BCTs using the guidelines developed by Michie et al [48]. For each article, the BCTs identified were documented and categorized using supporting textual evidence. They were coded only once per article regardless of how many times they came up in the text. Disagreements about including a BCT were resolved through discussion. If there was uncertainty about whether a BCT was present, it was coded as absent. Excel (Microsoft Corp) was used to compare the reviewers’ independent BCT coding and generate an overall descriptive synthesis of the BCTs identified. The BCTs were summarized by study and BCT cluster.
Statistical Analysis
Meta-analyses were conducted to estimate the size of the effect of the digital interventions for young adult CU on outcomes of interest at the posttreatment and follow-up assessments compared with control or alternative intervention conditions. The outcome variables considered were (1) CU frequency and other CU outcome variables (eg, quantity of cannabis used and abstinence) at baseline and the posttreatment time point or follow-up measured using standardized instruments of self-reported CU (eg, the timeline followback [TLFB] method) [66] and (2) cannabis-related negative consequences measured using standardized instruments (eg, the Marijuana Problems Scale) [67].
Under our systematic review protocol, ≥2 studies were needed for a meta-analysis. On the basis of previous systematic reviews and meta-analyses in the field of digital CU interventions [31,42-45], we expected between-study heterogeneity regarding outcome assessment. To minimize heterogeneity, we chose to pool studies with similar outcomes of interest based on four criteria: (1) definition of outcome (eg, CU frequency, quantity consumed, and abstinence), (2) type of outcome variable (eg, days of CU in the previous 90 days, days high per week in the previous 30 days, and number of CU events in the previous month) and measure (ie, instruments or scales), (3) use of validated instruments, and (4) posttreatment or follow-up time points (eg, 2 weeks or 1 month after the baseline or 3, 6, and 12 months after the baseline).
Only articles that reported sufficient statistics to compute a valid effect size with 95% CIs were included in the meta-analyses. In the case of articles that were not independent (ie, more than one published article reporting data from the same clinical trial), only 1 was included, and it was represented only once in the meta-analysis for a given outcome variable regardless of whether the data used to compute the effect size were extracted from the original paper or a secondary analysis paper. We made sure that the independence of the studies included in the meta-analysis of each outcome was respected. In the case of studies that had more than one comparator, we used the effect size for each comparison between the intervention and control groups.
Meta-analyses were conducted only for mean differences based on the change from baseline in CU frequency at 3 months after the baseline as measured using the number of self-reported days of use in the previous month. As the true value of the estimated effect size for outcome variables might vary across different trials and samples, we used a random-effects model given that the studies retained did not have identical target populations. The random-effects model incorporates between-study variation in the study weights and estimated effect size [68]. In addition, statistical heterogeneity across studies was assessed using I2, which measures the proportion of heterogeneity to the total observed dispersion; 25% was considered low, 50% was considered moderate, and 75% was considered high [69]. Because only 3 studies were included in the meta-analysis [70-72], publication bias could not be assessed. All analyses were completed using Stata (version 18; StataCorp) [73].
Risk-of-Bias Assessment
The risk of bias (RoB) of the included RCTs was assessed using the Cochrane RoB 2 tool at the outcome level [74]. Each distinct risk domain (ie, randomization process, deviations from the intended intervention, missing outcome data, measurement of the outcome, and selection of the reported results) was assessed as “low,” “some concerns,” or “high” based on the RoB 2 criteria. In total, 2 reviewers (GC and BV) conducted the assessments independently. Disagreements were discussed, and if not resolved consensually by the 2, the matter was left for a third reviewer (GF) to settle. The assessments were summarized by risk domain and outcome and converted into figures using the RoB visualization tool robvis [75].
Results
Search Results
The database search generated a total of 13,232 citations, of which 7822 (59.11%) were from the initial search on March 18, 2020, and 2805 (21.2%) and 2605 (19.69%) were from the updates on October 13, 2021, and February 13, 2023, respectively. Figure 1 presents the PRISMA study flow diagram [76]. Of the 6606 unique records, 6484 (98.15%) were excluded based on title and abstract screening. Full texts of the remaining 1.85% (122/6606) of the records were examined, as were those of 25 more reports found through hand searching. Of these 147 records, 128 (87.1%) were excluded after 3 rounds of full-text screening. Of these 128 records, 39 (30.5%) were excluded for not being empirical research articles (eg, research protocols). Another 28.1% (36/128) were excluded for not meeting our definition of digital CU intervention. The remaining records were excluded for reasons that occurred with a frequency of ≤14%, including young adults not being the target population and the study not meeting our study design criteria (ie, RCT, cluster RCT, or pilot RCT). Excluded studies and reasons for exclusion are listed in Multimedia Appendix 4. Finally, 19 articles detailing the results of 19 original studies were included.
Description of Studies
Study Characteristics
Multimedia Appendix 5 [70-72,77-92] describes the general characteristics of the 19 included studies. The studies were published between 2010 and 2023, with 58% (11/19) published in 2018 or later. A total of 53% (10/19) of the studies were conducted in the United States [77-86], 11% (2/19) were conducted in Canada [87,88], 11% (2/19) were conducted in Australia [71,89], 11% (2/19) were conducted in Germany [72,90], 11% (2/19) were conducted in Switzerland [70,91], and 5% (1/19) were conducted in Sweden [92]. A total of 79% (15/19) were RCTs [70-72,77,79,81-83,86-92], and 21% (4/19) were pilot RCTs [78,80,84,85].
Participant Characteristics
The studies enrolled a total of 6710 participants—3229 (48.1%) in the experimental groups, 3358 (50%) in the control groups, and the remaining 123 (1.8%) from 1 study [82] where participant allocation to the intervention condition was not reported. Baseline sample sizes ranged from 49 [81] to 1292 [72] (mean 352.89, SD 289.50), as shown in Multimedia Appendix 5. Participant mean ages ranged from 18.03 (SD 0.31) [79] to 35.3 (SD 12.6) years [88], and the proportion of participants who identified as female ranged from 24.7% [91] to 84.1% [80].
Of the 19 included studies, 10 (53%) targeted adults aged ≥18 years, of which 7 (70%) studies focused on adults who had engaged in past-month CU [70,71,80,84,85,90,91], 2 (20%) studies included adults who wished to reduce or cease CU [72,89], and 1 (10%) study focused on noncollege adults with a moderate risk associated with CU [88]. Sinadinovic et al [92] targeted young adults aged ≥16 years who had used cannabis at least once a week in the previous 6 months. The remaining 8 studies targeted college or university students (aged ≥17 y) specifically, of which 7 (88%) studies focused solely on students who reported using cannabis [78,79,81-83,86,87] and 1 (12%) study focused solely on students who did not report past-month CU (ie, abstainers) [77].
Intervention Characteristics
The 19 included studies assessed nine different digital interventions: (1) 5 (26%) evaluated Marijuana eCHECKUP TO GO (e-TOKE), a commercially available electronic intervention used at colleges throughout the United States and Canada [77,78,81-83]; (2) 2 (11%) examined the internationally known CANreduce program [70,91]; (3) 2 (11%) evaluated the German Quit the Shit program [72,90]; (4) 2 (11%) assessed a social media–delivered, physical activity–focused cannabis intervention [84,85]; (5) 1 (5%) investigated the Swedish Cannabishjälpen intervention [92]; (6) 1 (5%) evaluated the Australian Grassessment: Evaluate Your Use of Cannabis website program [89]; (7) 1 (5%) assessed the Canadian Ma réussite, mon choix intervention [87]; (8) 1 (5%) examined the Australian Reduce Your Use: How to Break the Cannabis Habit program [71]; and (9) 4 (21%) each evaluated a unique no-name intervention described as a personalized feedback intervention (PFI) [79,80,86,88]. Detailed information regarding the characteristics of all interventions as reported in each included study is provided in Multimedia Appendix 6 [70-72,77-113] and summarized in the following paragraphs.
In several studies (8/19, 42%), the interventions were designed to support cannabis users in reducing or ceasing their consumption [70,72,80,87,89-92]. In 37% (7/19) of the studies, the interventions aimed at reducing both CU and cannabis-related consequences [79,81-85,88]. Other interventions focused on helping college students think carefully about the decision to use cannabis [77,78] and on reducing either cannabis-related problems among undergraduate students [86] or symptoms associated with CU disorder in young adults [71].
In 26% (5/19) of the studies, theory was used to inform intervention design along with a clear rationale for theory use. Of these 5 articles, only 1 (20%) [87] reported using a single theory of behavior change, the theory of planned behavior [114]. A total of 21% (4/19) of the studies selected only constructs of theories (or models) for their intervention design. Of these 4 studies, 2 (50%) evaluated the same intervention [72,90], which focused on principles of self-regulation and self-control theory [93]; 1 (25%) [70] used the concept of adherence-focused guidance enhancement based on the supportive accountability model of guidance [94]; and 1 (25%) [71] reported that intervention design was guided by the concept of self-behavioral management.
The strategies (or approaches) used in the delivery of the digital interventions were discussed in greater detail in 84% (16/19) of the articles [70-72,79-81,83-92]. Many of these articles (9/19, 47%) reported using a combination of approaches based on CBT or motivational interviewing (MI) [70,71,79,83-85,90-92]. PFIs were also often mentioned as an approach to inform intervention delivery [7,71,79,86-88].
More than half (13/19, 68%) of all the digital interventions were asynchronous and based on a self-guided approach without support from a counselor or therapist. The study by Côté et al [87] evaluated the efficacy of a web-based tailored intervention focused on reinforcing a positive attitude toward and a sense of control over cannabis abstinence through psychoeducational messages delivered by a credible character in short video clips and personalized reinforcement messages. Lee et al [79] evaluated a brief, web-based personalized feedback selective intervention based on the PFI approach pioneered by Marlatt et al [95] for alcohol use prevention and on the MI approach described by Miller and Rollnick [96]. Similarly, Rooke et al [71] combined principles of MI and CBT to develop a web-based intervention delivered via web modules, which were informed by previous automated feedback interventions targeting substance use. The study by Copeland et al [89] assessed the short-term effectiveness of Grassessment: Evaluate Your Use of Cannabis, a brief web-based, self-complete intervention based on motivational enhancement therapy that included personalized feedback messages and psychoeducational material. In the studies by Buckner et al [80], Cunningham et al [88], and Walukevich-Dienst et al [86], experimental groups received a brief web-based PFI available via a computer. A total of 16% (3/19) of the studies [77,78,82] applied a program called the Marijuana eCHECKUP TO GO (e-TOKE) for Universities and Colleges, which was presented as a web-based, norm-correcting, brief preventive and intervention education program designed to prompt self-reflection on consequences and consideration of decreasing CU among students. Riggs et al [83] developed and evaluated an adapted version of e-TOKE that provided participants with university-specific personalized feedback and normative information based on protective behavioral strategies for CU [97]. Similarly, Goodness and Palfai [81] tested the efficacy of eCHECKUP TO GO-cannabis, a modified version of e-TOKE combining personalized feedback, norm correction, and a harm and frequency reduction strategy where a “booster” session was provided at 3 months to allow participants to receive repeated exposure to the intervention.
In the remaining 32% (6/19) of the studies, which examined 4 different interventions, the presence of a therapist guide was reported. The intervention evaluated by Sinadinovic et al [92] combined principles of psychoeducation, MI, and CBT organized into 13 web-based modules and a calendar involving therapist guidance, recommendations, and personal feedback. In total, 33% (2/6) of these studies evaluated a social media–delivered intervention with e-coaches that combined principles of MI and CBT and a harm reduction approach for risky CU [84,85]. Schaub et al [91] evaluated the efficacy of CANreduce, a web-based self-help intervention based on both MI and CBT approaches, using automated motivational and feedback emails, chat with a counselor, and web-based psychoeducational modules. Similarly, Baumgartner et al [70] investigated the effectiveness of CANreduce 2.0, a modified version of CANreduce, using semiautomated motivational and adherence-focused guidance-based email feedback with or without a personal online coach. The studies by Tossman et al [72] and Jonas et al [90] used a solution-focused approach and MI to evaluate the effectiveness of the German Quit the Shit web-based program that involves weekly feedback provided by counselors.
In addition to using different intervention strategies or approaches, the interventions were diverse in terms of the duration and frequency of the program (eg, web-based activities, sessions, or modules). Of the 12 articles that provided details in this regard, 2 (17%) on the same intervention described it as a brief 20- to 45-minute web-based program [77,78], 2 (17%) on 2 different interventions reported including 1 or 2 modules per week for a duration of 6 weeks [71,92], and 7 (58%) on 4 different interventions described them as being available over a longer period ranging from 6 weeks to 3 months [70,72,79,84,85,87,90,91].
Comparator Types
A total of 42% (8/19) of the studies [72,77-80,85,87,92] used a passive comparator only, namely, a waitlist control group (Multimedia Appendix 5). A total of 26% (5/19) of the studies used an active comparator only where participants were provided with minimal general health feedback regarding recommended guidelines for sleep, exercise, and nutrition [81,82]; strategies for healthy stress management [83]; educational materials about risky CU [88]; or access to a website containing information about cannabis [71]. In another 21% (4/19) of the studies, which used an active comparator, participants received the same digital intervention minus a specific component: a personal web-based coach [70], extended personalized feedback [89], web-based chat counseling [91], or information on risks associated with CU [86]. A total of 21% (4/19) of the studies had more than one control group [70,84,90,91].
Outcome Variable Assessment and Summary of Main Findings of the Studies
Overview
The methodological characteristics and major findings of the included studies (N=19) are presented in Multimedia Appendix 7 [67,70-72,77-92,115-120] and summarized in the following sections for each outcome of interest in this review (ie, CU and cannabis-related consequences). Of the 19 studies, 11 (58%) were reported as efficacy trials [7,77,79,81-83,86-88,91,92], and 8 (42%) were reported as effectiveness trials [70-72,78,84,85,89,90].
Across all the included studies (19/19, 100%), participant attrition rates ranged from 1.6% at 1 month after the baseline [77,78] to 75.1% at the 3-month follow-up [70]. A total of 37% (7/19) of the studies assessed and reported results regarding user engagement [71,78,84,85,90-92] using different types of metrics. In one article on the Marijuana eCHECKUP TO GO (e-TOKE) web-based program [78], the authors briefly reported that participation was confirmed for 98.1% (158/161) of participants in the intervention group. In 11% (2/19) of the studies, which were on a similar social media–delivered intervention [84,85], user engagement was quantified by tallying the number of comments or posts and reactions (eg, likes and hearts) left by participants. In both studies [84,85], the intervention group, which involved a CU-related Facebook page, displayed greater interactions than the control groups, which involved a Facebook page unrelated to CU. One article [84] reported that 80% of participants in the intervention group posted at least once (range 0-60) and 50% posted at least weekly. In the other study [85], the results showed that intervention participants engaged (ie, posting or commenting or clicking reactions) on average 47.9 times each over 8 weeks. In total, 11% (2/19) of the studies [90,91] on 2 different web-based intervention programs, both consisting of web documentation accompanied by chat-based counseling, measured user engagement either by average duration or average number of chat sessions. Finally, 16% (3/19) of the studies [71,91,92], which involved 3 different web-based intervention programs, characterized user engagement by the mean number of web modules completed per participant. Overall, the mean number of web modules completed reported in these articles was quite similar: 3.9 out of 13 [92] and 3.2 [91] and 3.5 [71] out of 6.
Assessment of CU
As presented in Multimedia Appendix 7, the included studies differed in terms of how they assessed CU, although all used at least one self-reported measure of frequency. Most studies (16/19, 84%) measured frequency by days of use, including days of use in the preceding week [91] or 2 [80], days of use in the previous 30 [70-72,78,84-86,88-90] or 90 days [79,81,82], and days high per week [83]. Other self-reported measures of CU frequency included (1) number of CU events in the previous month [87,90], (2) cannabis initiation or use in the previous month (ie, yes or no) [77], and (3) days without CU in the previous 7 days [92]. In addition to measuring CU frequency, 42% (8/19) of the studies also assessed CU via self-reported measures of quantity used, including estimated grams consumed in the previous week [92] or 30 days [72,85,90] and the number of standard-sized joints consumed in the previous 7 days [91] or the previous month [70,71,89].
Of the 19 articles included, 10 (53%) [70-72,80,84-86,89,90,92] reported using a validated instrument to measure CU frequency or quantity, including the TLFB instrument [66] (n=9, 90% of the studies) and the Marijuana Use Form (n=1, 10% of the studies); 1 (10%) [79] reported using CU-related questions from an adaptation of the Global Appraisal of Individual Needs–Initial instrument [115]; and 30% (3/10) [81,82,91] reported using a questionnaire accompanied by a calendar or a diary of consumption. The 19 studies also differed with regard to their follow-up time measurements for assessing CU, ranging from 2 weeks after the baseline [80] to 12 months after randomization [90], although 12 (63%) of the studies included a 3-month follow-up assessment [70-72,79,81,82,84,85,88,90-92].
Of all studies assessing and reporting change in CU frequency from baseline to follow-up assessments (19/19, 100%), 47% (9/19) found statistically significant differences between the experimental and control groups [70-72,80,81,83,85,87,91]. Importantly, 67% (6/9) of these studies showed that participants in the experimental groups exhibited greater decreases in CU frequency 3 months following the baseline assessment compared with participants in the control groups [70-72,81,85,91], 22% (2/9) of the studies showed greater decreases in CU frequency at 6 weeks after the baseline assessment [71,83], 22% (2/9) of the studies showed greater decreases in CU frequency at 6 months following the baseline assessment [81,85], 11% (1/9) of the studies showed greater decreases in CU frequency at 2 weeks after the baseline [80], and 11% (1/9) of the studies showed greater decreases in CU frequency at 2 months after treatment [87].
In the study by Baumgartner et al [70], a reduction in CU days was observed in all groups, but the authors reported that the difference was statistically significant only between the intervention group with the service team and the control group (the reduction in the intervention group with social presence was not significant). In the study by Bonar et al [85], the only statistically significant difference between the intervention and control groups at the 3- and 6-month follow-ups involved total days of cannabis vaping in the previous 30 days. Finally, in the study by Buckner et al [80], the intervention group had less CU than the control group 2 weeks after the baseline; however, this was statistically significant only for participants with moderate or high levels of social anxiety.
Descriptive Summary of BCTs Used in Intervention Groups
After the 19 studies included in this review were coded, a total of 184 individual BCTs targeting CU in young adults were identified. Of these 184 BCTs, 133 (72.3%) were deemed to be present beyond a reasonable doubt, and 51 (27.7%) were deemed to be present in all probability. Multimedia Appendix 8 [48,70-72,77-92] presents all the BCTs coded for each included study summarized by individual BCT and BCT cluster.
The 184 individual BCTs coded covered 38% (35/93) of the BCTs listed in the BCTTv1 [48]. The number of individual BCTs identified per study ranged from 5 to 19, with two-thirds of the 19 studies (12/19, 63%) using ≤9 BCTs (mean 9.68). As Multimedia Appendix 8 shows, at least one BCT fell into 13 of the 16 possible BCT clusters. The most frequent clusters were feedback monitoring, natural consequences, goal planning, and comparison of outcomes.
The most frequently coded BCTs were (1) feedback on behavior (BCT 2.2; 17/19, 89% of the studies; eg, “Once a week, participants receive detailed feedback by their counselor on their entries in diary and exercises. Depending on the involvement of each participant, up to seven feedbacks are given” [90]), (2) social support (unspecified) (BCT 3.1; 15/19, 79% of the studies; eg, “The website also features [...] blogs from former cannabis users, quick assist links, and weekly automatically generated encouragement emails” [71]), and (3) pros and cons (BCT 9.2; 14/19, 74% of the studies; eg, “participants are encouraged to state their personal reasons for and against their cannabis consumption, which they can review at any time, so they may reflect on what they could gain by successfully completing the program” [70]). Other commonly identified BCTs included social comparison (BCT 6.2; 12/19, 63% of the studies) and information about social and environmental consequences (BCT 5.3; 11/19, 58% of the studies), followed by problem solving (BCT 2.1; 10/19, 53% of the studies) and information about health consequences (BCT 5.1; 10/19, 53% of the studies).
RoB Assessment
Figure 2 presents the overall assessment of risk in each domain for all the included studies, whereas Figure 3 [70-72,77-92] summarizes the assessment of each study at the outcome level for each domain in the Cochrane RoB 2 [74].
Figure 2 shows that, of the included studies, 93% (27/29) were rated as having a “low” RoB arising from the randomization process (ie, selection bias) and 83% (24/29) were rated as having a “low” RoB due to missing data (ie, attrition bias). For bias due to deviations from the intended intervention (ie, performance bias), 72% (21/29) were rated as having a “low” risk, and for selective reporting of results, 59% (17/29) were rated as having a “low” risk. In the remaining domain regarding bias in measurement of the outcome (ie, detection bias), 48% (14/29) of the studies were deemed to present “some concerns,” mainly owing to the outcome assessment not being blinded (eg, self-reported outcome measure of CU). Finally, 79% (15/19) of the included studies were deemed to present “some concerns” or were rated as having a “high” RoB at the outcome level (Figure 3 [70-72,77-92]). The RoB assessment for CU and cannabis consequences of each included study is presented in Multimedia Appendix 9 [70-72,77-92].
Meta-Analysis Results
Due to several missing data points and despite contacting the authors, we were able to carry out only 1 meta-analysis of our primary outcome, CU frequency. Usable data were retrieved from only 16% (3/19) [70-72] of the studies included in this review. These 3 studies provided sufficient information to calculate an effect size, including mean differences based on change-from-baseline measurements and associated 95% CIs (or SE of the mean difference) and sample sizes per intervention and comparison conditions. The reasons for excluding the other 84% (16/19) of the studies included heterogeneity in outcome variables or measurements, inconsistent results, and missing data (Multimedia Appendix 10 [77-92]).
Figure 4 [70-72] illustrates the mean differences and associated 95% CIs of 3 unique RCTs [70-72] that provided sufficient information to allow for the measurement of CU frequency at 3 months after the baseline relative to a comparison condition in terms of the number of self-reported days of use in the previous month using the TLFB method. Overall, the synthesized effect of digital interventions for young adult cannabis users on CU frequency, as measured using days of use in the previous month, was −6.79 (95% CI −9.59 to −4.00). This suggests that digital CU interventions had a statistically significant effect (P<.001) on reducing CU frequency at the 3-month follow-up compared with the control conditions (both passive and active controls). The results of the meta-analysis also showed low between-study heterogeneity (I2=48.3%; P=.12) across the 3 included studies.
The samples of the 3 studies included in the meta-analysis varied in size from 225 to 1292 participants (mean 697.33, SD 444.11), and the mean age ranged from 24.7 to 31.88 years (mean 26.38, SD 3.58 years). These studies involved 3 different digital interventions and used different design approaches to assess intervention effectiveness. One study assessed the effectiveness of a web-based counseling program (ie, Quit the Shit) against a waitlist control [72], another examined the effectiveness of a fully self-guided web-based treatment program for CU and related problems (ie, Reduce Your Use: How to Break the Cannabis Habit) against a control condition website consisting of basic educational information on cannabis [71], and the third used a 3-arm RCT design to investigate whether the effectiveness of a minimally guided internet-based self-help intervention (ie, CANreduce 2.0) might be enhanced by implementing adherence-focused guidance and emphasizing the social presence factor of a personal e-coach [70].
Discussion
Summary of Principal Findings
The primary aim of this systematic review was to evaluate the effectiveness of digital interventions in addressing CU among community-living young adults. We included 19 randomized controlled studies representing 9 unique digital interventions aimed at preventing, reducing, or ceasing CU and evaluated the effects of 3 different digital interventions on CU. In summary, the 3 digital interventions included in the meta-analysis proved superior to control conditions in reducing the number of days of CU in the previous month at the 3-month follow-up.
Our findings are consistent with those of 2 previous meta-analyses by Olmos et al [43] and Tait et al [44] and with the findings of a recently published umbrella review of systematic reviews and meta-analyses of RCTs [123], all of which revealed a positive effect of internet- and computer-based interventions on CU. However, a recent systematic review and meta-analysis by Beneria et al [45] found that web-based CU interventions did not significantly reduce CU. Beneria et al [45] included studies with different intervention programs that targeted diverse population groups (both adolescents and young adults) and use of more than one substance (eg, alcohol and cannabis). In our systematic review, a more conservative approach was taken—we focused specifically on young adults and considered interventions targeting CU only. Although our results indicate that digital interventions hold great promise in terms of effectiveness, an important question that remains unresolved is whether there is an optimal exposure dose in terms of both duration and frequency that might be more effective. Among the studies included in this systematic review, interventions varied considerably in terms of the number of psychoeducational modules offered (from 2 to 13), time spent reviewing the material, and duration (from a single session to a 12-week spread period). Our results suggest that an intervention duration of at least 6 weeks yields better results.
Another important finding of this review is that, although almost half (9/19, 47%) of the included studies observed an intervention effect on CU frequency, none reported a statistically significant improvement in cannabis-related negative consequences, which may be considered a more distal indicator. More than half (10/19, 53%) of the included studies investigated this outcome. It seems normal to expect to find an effect on CU frequency given that reducing CU is often the primary objective of interventions and because the motivation of users’ is generally focused on changing consumption behavior. It is plausible to think that the change in behavior at the consumption level must be maintained over time before an effect on cannabis-related negative consequences can be observed. However, our results showed that, in all the included studies, cannabis-related negative consequences and change in behavior (CU frequency) were measured at the same time point, namely, 3 months after the baseline. Moreover, Grigsby et al [124] conducted a scoping review of risk and protective factors for CU and suggested that interventions to reduce negative CU consequences should prioritize multilevel methods or strategies “to attenuate the cumulative risk from a combination of psychological, contextual, and social influences.”
A secondary objective of this systematic review was to describe the active ingredients used in digital interventions for CU among young adults. The vast majority of the interventions were based on either a theory or an intervention approach derived from theories such as CBT, MI, and personalized feedback. From these theories and approaches stem behavior change strategies or techniques, commonly known as BCTs. Feedback on behavior, included in the feedback monitoring BCT cluster, was the most common BCT used in the included studies. This specific BCT appears to be a core strategy in behavior change interventions [125,126]. In their systematic review of remotely delivered alcohol or substance misuse interventions for adults, Howlett et al [53] found that feedback on behavior, problem solving, and goal setting were the most frequently used BCTs in the included studies. In addition, this research group noted that the most promising BCTs for alcohol misuse were avoidance/reducing exposure to cues for behavior, pros and cons, and self-monitoring of behavior, whereas 2 very promising strategies for substance misuse in general were problem solving and self-monitoring of behavior. In our systematic review, in addition to feedback on behavior, the 6 most frequently used BCTs in the included studies were social support, pros and cons, social comparison, problem solving, information about social and environmental consequences, and information about health consequences. Although pros and cons and problem solving were present in all 3 studies of digital interventions included in our meta-analysis, avoidance/reducing exposure to cues for behavior was reported in only 5% (1/19) of the articles, and feedback on behavior was more frequently used than self-monitoring of behavior. However, it should be noted that the review by Howlett et al [53] examined digital interventions for participants with alcohol or substance misuse problems, whereas in this review, we focused on interventions that targeted CU from a harm reduction perspective. In this light, avoidance/reducing exposure to cues for behavior may be a BCT better suited to populations with substance misuse problems. Lending support to this, a meta-regression by Garnett et al [127] and a Cochrane systematic review by Kaner et al [128] both found interventions that used behavior substitution and credible source to be associated with greater reduction in excessive alcohol consumption compared with interventions that used other BCTs.
Beyond the number and types of BCTs used, reflecting on the extent to which each BCT in a given intervention suits (or does not suit) the targeted determinants (ie, behavioral and environmental causes) is crucial for planning intervention programs [26]. It is important when designing digital CU interventions not merely to pick a combination of BCTs that have been associated with effectiveness. Rather, the active ingredients must fit the determinants that the interventionists seek to influence. For example, action planning would be more relevant as a BCT for young adults highly motivated and ready to take action on their CU than would pros and cons, which aims instead to bolster motivation. Given that more than half of all digital interventions are asynchronous and based on a self-guided approach and do not offer counselor or therapist support, a great deal of motivation is required to engage in intervention and behavior change. Therefore, it is essential that developers consider the needs and characteristics of the targeted population to tailor intervention strategies (ie, BCTs) for successful behavior change (eg, tailored to the participant’s stage of change). In most of the digital interventions included in this systematic review, personalization was achieved through feedback messages about CU regarding descriptive norms, motives, risks and consequences, and costs, among other things.
Despite the high number of recent studies conducted in the field of digital CU interventions, most of the included articles in our review (17/19, 89%) reported on the development and evaluation of web-based intervention programs. A new generation of health intervention modalities such as mobile apps and social media has drawn the attention of researchers in the past decade and is currently being evaluated. In this regard, the results from a recently published scoping review [129], which included 5 studies of mobile apps for nonmedical CU, suggested that these novel modes of intervention delivery demonstrated adequate feasibility and acceptability. Nevertheless, the internet remains a powerful and convenient medium for reaching young adults with digital interventions intended to support safe CU behaviors [123,130].
Quality of Evidence
The GRADE (Grading of Recommendations Assessment, Development, and Evaluation) approach [131-133] was used to assess the quality of the evidence reviewed. It was deemed to be moderate for the primary outcome of this review, that is, CU frequency in terms of days of use in the previous month (see the summary of evidence in Multimedia Appendix 11 [70,72]). The direction of evidence was broadly consistent—in all 3 RCT studies [70-72] included in the meta-analysis, participants who received digital CU interventions reduced their consumption compared with those who received no or minimal interventions. The 3 RCTs were similar in that they all involved a web-based, multicomponent intervention program aimed at reducing or ceasing CU. However, the interventions did differ or vary in terms of several characteristics, including the strategies used, content, frequency, and duration. Given the small number of studies included in the meta-analysis, we could not conclude with certainty which intervention components, if any, contributed to the effect estimate observed.
Although inconsistency, indirectness, and imprecision were not major issues in the body of evidence, we downgraded the evidence from high to moderate quality on account of RoB assessments at the outcome level. The 3 RCT studies included in the meta-analysis were rated as having “some concerns” of RoB, mainly due to lack of blinding, which significantly reduced our certainty relative to subjective outcomes (ie, self-reported measures of CU frequency). A positive feature of these digital intervention trials is that most procedures are fully automated, and so there was typically a low RoB regarding randomization procedures, allocation to different conditions, and intervention delivery. It is impossible to blind participants to these types of behavior change interventions, and although some researchers have made attempts to counter the impact of this risk, performance bias is an inescapable issue in RCT studies of this kind. Blinding of intervention providers was not an issue in the 3 RCTs included in the meta-analysis because outcome data collection was automated. However, this same automated procedure made it very difficult to ensure follow‐up. Consequently, attrition was another source of bias in these RCT studies [70-72]. The participants lost to follow-up likely stopped using the intervention. However, there is no way of determining whether these people would have benefited more or less than the completers if they had seen the trial through.
The 3 RCTs included in the meta-analysis relied on subjective self-reported measures of CU at baseline and follow‐up, which are subject to recall and social desirability bias. However, all 3 studies used a well-validated instrument of measurement to determine frequency of CU, the TLFB [66]. This is a widely used, subjective self-report tool for measuring frequency (or quantity) of substance use (or abstinence). It is considered a reliable measure of CU [134,135]. Finally, it should be pointed out that any potential bias related to self‐reported CU frequency would have affected both the intervention and control groups (particularly in cases in which control groups received cannabis‐related information), and thus, it was unlikely to account for differential intervention effects. Moreover, we found RoB due to selective reporting in some studies owing mainly to the absence of any reference to a protocol. Ultimately, these limitations may have biased the results of the meta-analysis. Consequently, future research is likely to further undermine our confidence in the effect estimate we observed and report considerably different estimates.
Strengths and Limitations
Our systematic review and meta-analysis has a number of strengths: (1) we included only randomized controlled studies to ensure that the included studies possessed a rigorous research design, (2) we focused specifically on cannabis (rather than combining multiple substances), (3) we assessed the effectiveness of 3 different digital interventions on CU frequency among community-living young adults, and (4) we performed an exhaustive synthesis and comparison of the BCTs used in the 9 digital interventions examined in the 19 studies included in our review based on the BCTTv1.
Admittedly, this systematic review and meta-analysis has limitations that should be recognized. First, although we searched a range of bibliographic databases, the review was limited to articles published in peer-reviewed journals in English or French. This may have introduced publication bias given that articles reporting positive effects are more likely to be published than those with negative or equivocal results. Consequently, the studies included in this review may have overrepresented the statistically significant effects of digital CU interventions.
Second, only a small number of studies were included in the meta-analyses because many studies did not provide adequate statistical information for calculating and synthesizing effect sizes, although significant efforts were made to contact the authors in case of missing data. Because of the small sample size used in the meta-analysis, the effect size estimates may not be highly reflective of the true effects of digital interventions on CU frequency among young adults. Furthermore, synthesizing findings across studies that evaluated different modalities of web-based intervention programs (eg, fully self-guided vs with therapist guidance) and types of intervention approaches (eg, CBT, MI, and personalized feedback) may have introduced bias in the meta-analytical results due to the heterogeneity of the included studies, although heterogeneity was controlled for using a random-effects model and our results indicated low between-study heterogeneity.
Third, we took various measures to ensure that BCT coding was carried out rigorously throughout the data extraction and analysis procedures: (1) all coders received training on how to use the BCTTv1; (2) all the included articles were read line by line so that coders became familiar with intervention descriptions before initiating BCT coding; (3) the intervention description of each included article was double coded after a pilot calibration exercise with all coders, and any disagreements regarding the presence or absence of a BCT were discussed and resolved with a third party; and (4) we contacted the article authors when necessary and possible for further details on the BCTs they used. However, incomplete reporting of intervention content is a recognized issue [136], which may have resulted in our coding BCTs incorrectly as present or absent. Reliably specifying the BCTs used in interventions allows their active ingredients to be identified, their evidence to be synthesized, and interventions to be replicated, thereby providing tangible guidance to programmers and researchers to develop more effective interventions.
Finally, although this review identified the BCTs used in digital interventions, our approach did not allow us to draw conclusions regarding their effectiveness. Coding BCTs simply as present or absent does not consider the frequency, intensity, and quality with which they were delivered. For example, it is unclear how many individuals should self‐monitor their CU. In addition, the quality of BCT implementation may be critical in digital interventions where different graphics and interface designs and the usability of the BCTs used can have considerable influence on the level of user engagement [137]. In the future, it may be necessary to develop new methods to evaluate the dosage of individual BCTs in digital health interventions and characterize their implementation quality to assess their effectiveness [128,138]. Despite its limitations, this review suggests that digital interventions represent a promising avenue for preventing, reducing, or ceasing CU among community-living young adults.
Conclusions
The results of this systematic review and meta-analysis lend support to the promise of digital interventions as an effective means of reducing recreational CU frequency among young adults. Despite the advent and popularity of smartphones, web-based interventions remain the most common mode of delivery for digital interventions. The active ingredients of digital interventions are varied and encompass a number of clusters of the BCTTv1, but a significant number of BCTs remain underused. Additional research is needed to further investigate the effectiveness of these interventions on CU and key outcomes at later time points. Finally, a detailed assessment of user engagement with digital interventions for CU and understanding which intervention components are the most effective remain important research gaps.
Acknowledgments
The authors would like to thank Bénédicte Nauche, Miguel Chagnon, and Paul Di Biase for their valuable support with the search strategy development, statistical analysis, and linguistic revision, respectively. This work was supported by the Ministère de la Santé et des Services sociaux du Québec as part of a broader study aimed at developing and evaluating a digital intervention for young adult cannabis users. Additional funding was provided by the Research Chair in Innovative Nursing Practices. The views and opinions expressed in this manuscript do not necessarily reflect those of these funding entities.
Abbreviations
- BCT
- behavior change technique
- BCTTv1
- Behavior Change Technique Taxonomy version 1
- CBT
- cognitive behavioral therapy
- CU
- cannabis use
- GRADE
- Grading of Recommendations Assessment, Development, and Evaluation
- MI
- motivational interviewing
- PICO
- population, intervention, comparator, and outcome
- PFI
- personalized feedback intervention
- PRISMA
- Preferred Reporting Items for Systematic Reviews and Meta-Analyses
- RCT
- randomized controlled trial
- RoB
- risk of bias
- TLFB
- Timeline Followback
Untitled section
Untitled section
Data Availability
The data sets generated and analyzed during this study are available from the corresponding author on reasonable request.
Footnotes
Footnote Group
References
Untitled section
References
- 1.Arnett JJ. The developmental context of substance use in emerging adulthood. J Drug Issues. 2005;35(2):235–54. doi: 10.1177/002204260503500202.
- 2.Stockings E, Hall WD, Lynskey M, Morley KI, Reavley N, Strang J, Patton G, Degenhardt L. Prevention, early intervention, harm reduction, and treatment of substance use in young people. Lancet Psychiatry. 2016 Mar;3(3):280–96. doi: 10.1016/S2215-0366(16)00002-X.S2215-0366(16)00002-X
- 3.ElSohly MA, Chandra S, Radwan M, Majumdar CG, Church JC. A comprehensive review of cannabis potency in the United States in the last decade. Biol Psychiatry Cogn Neurosci Neuroimaging. 2021 Jun;6(6):603–6. doi: 10.1016/j.bpsc.2020.12.016.S2451-9022(21)00022-7
- 4.Fischer B, Robinson T, Bullen C, Curran V, Jutras-Aswad D, Medina-Mora ME, Pacula RL, Rehm J, Room R, van den Brink W, Hall W. Lower-Risk Cannabis Use Guidelines (LRCUG) for reducing health harms from non-medical cannabis use: a comprehensive evidence and recommendations update. Int J Drug Policy. 2022 Jan;99:103381. doi: 10.1016/j.drugpo.2021.103381. https://linkinghub.elsevier.com/retrieve/pii/S0955-3959(21)00286-3 .S0955-3959(21)00286-3
- 5.Rotermann M. What has changed since cannabis was legalized? Health Rep. 2020 Feb 19;31(2):11–20. doi: 10.25318/82-003-x202000200002-eng. http://www.statcan.gc.ca/pub/82-003-x/2020002/article/00002-eng.pdf .82-003-X202000200002
- 6.Degenhardt L, Stockings E, Patton G, Hall WD, Lynskey M. The increasing global health priority of substance use in young people. Lancet Psychiatry. 2016 Mar;3(3):251–64. doi: 10.1016/S2215-0366(15)00508-8.S2215-0366(15)00508-8
- 7.Buckner JD, Bonn-Miller MO, Zvolensky MJ, Schmidt NB. Marijuana use motives and social anxiety among marijuana-using young adults. Addict Behav. 2007 Oct;32(10):2238–52. doi: 10.1016/j.addbeh.2007.04.004. https://europepmc.org/abstract/MED/17478056 .S0306-4603(07)00091-3
- 8.Carliner H, Brown QL, Sarvet AL, Hasin DS. Cannabis use, attitudes, and legal status in the U.S.: a review. Prev Med. 2017 Nov;104:13–23. doi: 10.1016/j.ypmed.2017.07.008. https://europepmc.org/abstract/MED/28705601 .S0091-7435(17)30255-4
- 9.World drug report 2020. United Nations Office on Drugs and Crime. 2020. [2023-11-28]. https://wdr.unodc.org/wdr2020/index2020.html .
- 10.National Academies of Sciences, Engineering, and Medicine. Health and Medicine Division. Board on Population Health and Public Health Practice. Committee on the Health Effects of Marijuana: An Evidence Review and Research Agenda . The Health Effects of Cannabis and Cannabinoids: The Current State of Evidence and Recommendations for Research. Washington, DC: The National Academies Press; 2017.
- 11.Hall WD, Patton G, Stockings E, Weier M, Lynskey M, Morley KI, Degenhardt L. Why young people's substance use matters for global health. Lancet Psychiatry. 2016 Mar;3(3):265–79. doi: 10.1016/S2215-0366(16)00013-4.S2215-0366(16)00013-4
- 12.Cohen K, Weizman A, Weinstein A. Positive and negative effects of cannabis and cannabinoids on health. Clin Pharmacol Ther. 2019 May;105(5):1139–47. doi: 10.1002/cpt.1381.
- 13.Memedovich KA, Dowsett LE, Spackman E, Noseworthy T, Clement F. The adverse health effects and harms related to marijuana use: an overview review. CMAJ Open. 2018 Aug 16;6(3):E339–46. doi: 10.9778/cmajo.20180023. http://cmajopen.ca/cgi/pmidlookup?view=long&pmid=30115639 .6/3/E339
- 14.Teeters JB, Armstrong NM, King SA, Hubbard SM. A randomized pilot trial of a mobile phone-based brief intervention with personalized feedback and interactive text messaging to reduce driving after cannabis use and riding with a cannabis impaired driver. J Subst Abuse Treat. 2022 Nov;142:108867. doi: 10.1016/j.jsat.2022.108867. https://europepmc.org/abstract/MED/36007434 .S0740-5472(22)00149-0
- 15.Chan GC, Becker D, Butterworth P, Hines L, Coffey C, Hall W, Patton G. Young-adult compared to adolescent onset of regular cannabis use: a 20-year prospective cohort study of later consequences. Drug Alcohol Rev. 2021 May;40(4):627–36. doi: 10.1111/dar.13239.
- 16.Hall W, Stjepanović D, Caulkins J, Lynskey M, Leung J, Campbell G, Degenhardt L. Public health implications of legalising the production and sale of cannabis for medicinal and recreational use. Lancet. 2019 Oct 26;394(10208):1580–90. doi: 10.1016/S0140-6736(19)31789-1.S0140-6736(19)31789-1
- 17.The health and social effects of nonmedical cannabis use. World Health Organization. 2016. [2023-11-28]. https://apps.who.int/iris/handle/10665/251056 .
- 18.Boumparis N, Loheide-Niesmann L, Blankers M, Ebert DD, Korf D, Schaub MP, Spijkerman R, Tait RJ, Riper H. Short- and long-term effects of digital prevention and treatment interventions for cannabis use reduction: a systematic review and meta-analysis. Drug Alcohol Depend. 2019 Jul 01;200:82–94. doi: 10.1016/j.drugalcdep.2019.03.016. https://linkinghub.elsevier.com/retrieve/pii/S0376-8716(19)30149-8 .S0376-8716(19)30149-8
- 19.Jutras-Aswad D, Le Foll B, Bruneau J, Wild TC, Wood E, Fischer B. Thinking beyond legalization: the case for expanding evidence-based options for cannabis use disorder treatment in Canada. Can J Psychiatry. 2019 Feb;64(2):82–7. doi: 10.1177/0706743718790955. https://europepmc.org/abstract/MED/30033757 .
- 20.Garnett CV, Crane D, Brown J, Kaner EF, Beyer FR, Muirhead CR, Hickman M, Beard E, Redmore J, de Vocht F, Michie S. Behavior change techniques used in digital behavior change interventions to reduce excessive alcohol consumption: a meta-regression. Ann Behav Med. 2018 May 18;52(6):530–43. doi: 10.1093/abm/kax029. https://europepmc.org/abstract/MED/29788261 .4827558
- 21.Glanz K, Rimer BK, Viswanath K. Health Behavior: Theory, Research, and Practice, 5th Edition. Hoboken, NJ: Jossey-Bass; 2015. Jul,
- 22.Prestwich A, Webb TL, Conner M. Using theory to develop and test interventions to promote changes in health behaviour: evidence, issues, and recommendations. Curr Opin Psychol. 2015 Oct;5:1–5. doi: 10.1016/j.copsyc.2015.02.011.
- 23.Webb TL, Sniehotta FF, Michie S. Using theories of behaviour change to inform interventions for addictive behaviours. Addiction. 2010 Nov;105(11):1879–92. doi: 10.1111/j.1360-0443.2010.03028.x.ADD3028
- 24.Cilliers F, Schuwirth L, van der Vleuten C. Health behaviour theories: a conceptual lens to explore behaviour change. In: Cleland J, Durning SJ, editors. Researching Medical Education. Hoboken, NJ: Wiley; 2015.
- 25.Davis R, Campbell R, Hildon Z, Hobbs L, Michie S. Theories of behaviour and behaviour change across the social and behavioural sciences: a scoping review. Health Psychol Rev. 2015;9(3):323–44. doi: 10.1080/17437199.2014.941722. https://europepmc.org/abstract/MED/25104107 .
- 26.Eldredge LK, Markham CM, Ruiter RA, Fernández ME, Kok G, Parcel GS. Planning Health Promotion Programs: An Intervention Mapping Approach, 4th Edition. Hoboken, NJ: John Wiley & Sons; 2016. Feb,
- 27.Marlatt GA, Blume AW, Parks GA. Integrating harm reduction therapy and traditional substance abuse treatment. J Psychoactive Drugs. 2001;33(1):13–21. doi: 10.1080/02791072.2001.10400463.
- 28.Adams A, Ferguson M, Greer AM, Burmeister C, Lock K, McDougall J, Scow M, Buxton JA. Guideline development in harm reduction: considerations around the meaningful involvement of people who access services. Drug Alcohol Depend Rep. 2022 Aug 12;4:100086. doi: 10.1016/j.dadr.2022.100086. https://linkinghub.elsevier.com/retrieve/pii/S2772-7246(22)00061-0 .S2772-7246(22)00061-0
- 29.Davis ML, Powers MB, Handelsman P, Medina JL, Zvolensky M, Smits JA. Behavioral therapies for treatment-seeking cannabis users: a meta-analysis of randomized controlled trials. Eval Health Prof. 2015 Mar;38(1):94–114. doi: 10.1177/0163278714529970. https://europepmc.org/abstract/MED/24695072 .0163278714529970
- 30.Gates PJ, Sabioni P, Copeland J, Le Foll B, Gowing L. Psychosocial interventions for cannabis use disorder. Cochrane Database Syst Rev. 2016 May 05;2016(5):CD005336. doi: 10.1002/14651858.CD005336.pub4. https://europepmc.org/abstract/MED/27149547 .
- 31.Halladay J, Scherer J, MacKillop J, Woock R, Petker T, Linton V, Munn C. Brief interventions for cannabis use in emerging adults: a systematic review, meta-analysis, and evidence map. Drug Alcohol Depend. 2019 Nov 01;204:107565. doi: 10.1016/j.drugalcdep.2019.107565.S0376-8716(19)30342-4
- 32.Imtiaz S, Roerecke M, Kurdyak P, Samokhvalov AV, Hasan OS, Rehm J. Brief interventions for cannabis use in healthcare settings: systematic review and meta-analyses of randomized trials. J Addict Med. 2020;14(1):78–88. doi: 10.1097/ADM.0000000000000527.01271255-202002000-00011
- 33.Standeven LR, Scialli A, Chisolm MS, Terplan M. Trends in cannabis treatment admissions in adolescents/young adults: analysis of TEDS-A 1992 to 2016. J Addict Med. 2020;14(4):e29–36. doi: 10.1097/ADM.0000000000000586.
- 34.Montanari L, Guarita B, Mounteney J, Zipfel N, Simon R. Cannabis use among people entering drug treatment in europe: a growing phenomenon? Eur Addict Res. 2017;23(3):113–21. doi: 10.1159/000475810. doi: 10.1159/000475810.000475810
- 35.Kerridge BT, Mauro PM, Chou SP, Saha TD, Pickering RP, Fan AZ, Grant BF, Hasin DS. Predictors of treatment utilization and barriers to treatment utilization among individuals with lifetime cannabis use disorder in the United States. Drug Alcohol Depend. 2017 Dec 01;181:223–8. doi: 10.1016/j.drugalcdep.2017.09.032. https://europepmc.org/abstract/MED/29107786 .S0376-8716(17)30525-2
- 36.Gates P, Copeland J, Swift W, Martin G. Barriers and facilitators to cannabis treatment. Drug Alcohol Rev. 2012 May;31(3):311–9. doi: 10.1111/j.1465-3362.2011.00313.x.
- 37.Hammarlund RA, Crapanzano KA, Luce L, Mulligan L, Ward KM. Review of the effects of self-stigma and perceived social stigma on the treatment-seeking decisions of individuals with drug- and alcohol-use disorders. Subst Abuse Rehabil. 2018 Nov 23;9:115–36. doi: 10.2147/SAR.S183256. https://europepmc.org/abstract/MED/30538599 .sar-9-115
- 38.Bedrouni W. On the use of digital technologies to reduce the public health impacts of cannabis legalization in Canada. Can J Public Health. 2018 Dec;109(5-6):748–51. doi: 10.17269/s41997-018-0117-7. https://europepmc.org/abstract/MED/30097890 .10.17269/s41997-018-0117-7
- 39.Perski O, Hébert ET, Naughton F, Hekler EB, Brown J, Businelle MS. Technology-mediated just-in-time adaptive interventions (JITAIs) to reduce harmful substance use: a systematic review. Addiction. 2022 May;117(5):1220–41. doi: 10.1111/add.15687. https://europepmc.org/abstract/MED/34514668 .
- 40.Kazemi DM, Borsari B, Levine MJ, Li S, Lamberson KA, Matta LA. A systematic review of the mHealth interventions to prevent alcohol and substance abuse. J Health Commun. 2017 May;22(5):413–32. doi: 10.1080/10810730.2017.1303556. https://europepmc.org/abstract/MED/28394729 .
- 41.Nesvåg S, McKay JR. Feasibility and effects of digital interventions to support people in recovery from substance use disorders: systematic review. J Med Internet Res. 2018 Aug 23;20(8):e255. doi: 10.2196/jmir.9873. https://www.jmir.org/2018/8/e255/ v20i8e255
- 42.Hoch E, Preuss UW, Ferri M, Simon R. Digital interventions for problematic cannabis users in non-clinical settings: findings from a systematic review and meta-analysis. Eur Addict Res. 2016;22(5):233–42. doi: 10.1159/000445716. doi: 10.1159/000445716.000445716
- 43.Olmos A, Tirado-Muñoz J, Farré M, Torrens M. The efficacy of computerized interventions to reduce cannabis use: a systematic review and meta-analysis. Addict Behav. 2018 Apr;79:52–60. doi: 10.1016/j.addbeh.2017.11.045.S0306-4603(17)30453-7
- 44.Tait RJ, Spijkerman R, Riper H. Internet and computer based interventions for cannabis use: a meta-analysis. Drug Alcohol Depend. 2013 Dec 01;133(2):295–304. doi: 10.1016/j.drugalcdep.2013.05.012.S0376-8716(13)00187-7
- 45.Beneria A, Santesteban-Echarri O, Daigre C, Tremain H, Ramos-Quiroga JA, McGorry PD, Alvarez-Jimenez M. Online interventions for cannabis use among adolescents and young adults: systematic review and meta-analysis. Early Interv Psychiatry. 2022 Aug;16(8):821–44. doi: 10.1111/eip.13226.
- 46.Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. Int J Nurs Stud. 2013 May;50(5):587–92. doi: 10.1016/j.ijnurstu.2012.09.010. https://core.ac.uk/reader/12828016?utm_source=linkout .S0020-7489(12)00306-9
- 47.Michie S, Abraham C, Eccles MP, Francis JJ, Hardeman W, Johnston M. Strengthening evaluation and implementation by specifying components of behaviour change interventions: a study protocol. Implement Sci. 2011 Feb 07;6:10. doi: 10.1186/1748-5908-6-10. https://implementationscience.biomedcentral.com/articles/10.1186/1748-5908-6-10 .1748-5908-6-10
- 48.Michie S, Richardson M, Johnston M, Abraham C, Francis J, Hardeman W, Eccles MP, Cane J, Wood CE. The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: building an international consensus for the reporting of behavior change interventions. Ann Behav Med. 2013 Aug;46(1):81–95. doi: 10.1007/s12160-013-9486-6. https://core.ac.uk/reader/191129821?utm_source=linkout .
- 49.Michie S, Johnston M, Francis J, Hardeman W, Eccles M. From theory to intervention: mapping theoretically derived behavioural determinants to behaviour change techniques. Appl Psychol. 2008 Oct;57(4):660–80. doi: 10.1111/j.1464-0597.2008.00341.x.
- 50.Scott C, de Barra M, Johnston M, de Bruin M, Scott N, Matheson C, Bond C, Watson MC. Using the behaviour change technique taxonomy v1 (BCTTv1) to identify the active ingredients of pharmacist interventions to improve non-hospitalised patient health outcomes. BMJ Open. 2020 Sep 15;10(9):e036500. doi: 10.1136/bmjopen-2019-036500. https://bmjopen.bmj.com/lookup/pmidlookup?view=long&pmid=32933960 .bmjopen-2019-036500
- 51.Dombrowski SU, Sniehotta FF, Avenell A, Johnston M, MacLennan G, Araújo-Soares V. Identifying active ingredients in complex behavioural interventions for obese adults with obesity-related co-morbidities or additional risk factors for co-morbidities: a systematic review. Health Psychol Rev. 2012;6(1):7–32. doi: 10.1080/17437199.2010.513298.
- 52.Michie S, Abraham C, Whittington C, McAteer J, Gupta S. Effective techniques in healthy eating and physical activity interventions: a meta-regression. Health Psychol. 2009 Nov;28(6):690–701. doi: 10.1037/a0016136.2009-20990-006
- 53.Howlett N, García-Iglesias J, Bontoft C, Breslin G, Bartington S, Freethy I, Huerga-Malillos M, Jones J, Lloyd N, Marshall T, Williams S, Wills W, Brown K. A systematic review and behaviour change technique analysis of remotely delivered alcohol and/or substance misuse interventions for adults. Drug Alcohol Depend. 2022 Oct 01;239:109597. doi: 10.1016/j.drugalcdep.2022.109597. https://linkinghub.elsevier.com/retrieve/pii/S0376-8716(22)00334-9 .S0376-8716(22)00334-9
- 54.Higgins JP, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA. Cochrane Handbook for Systematic Reviews of Interventions Version 6.4. London, UK: The Cochrane Collaboration; 2023.
- 55.Page MJ, Moher D, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, Shamseer L, Tetzlaff JM, Akl EA, Brennan SE, Chou R, Glanville J, Grimshaw JM, Hróbjartsson A, Lalu MM, Li T, Loder EW, Mayo-Wilson E, McDonald S, McGuinness LA, Stewart LA, Thomas J, Tricco AC, Welch VA, Whiting P, McKenzie JE. PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews. BMJ. 2021 Mar 29;372:n160. doi: 10.1136/bmj.n160. http://www.bmj.com/lookup/pmidlookup?view=long&pmid=33781993 .
- 56.McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS peer review of electronic search strategies: 2015 guideline statement. J Clin Epidemiol. 2016 Jul;75:40–6. doi: 10.1016/j.jclinepi.2016.01.021. https://linkinghub.elsevier.com/retrieve/pii/S0895-4356(16)00058-5 .S0895-4356(16)00058-5
- 57.Halladay J, Petker T, Fein A, Munn C, MacKillop J. Brief interventions for cannabis use in emerging adults: protocol for a systematic review, meta-analysis, and evidence map. Syst Rev. 2018 Jul 25;7(1):106. doi: 10.1186/s13643-018-0772-z. https://systematicreviewsjournal.biomedcentral.com/articles/10.1186/s13643-018-0772-z .10.1186/s13643-018-0772-z
- 58.Michie S, van Stralen MM, West R. The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci. 2011 Apr 23;6:42. doi: 10.1186/1748-5908-6-42. https://implementationscience.biomedcentral.com/articles/10.1186/1748-5908-6-42 .1748-5908-6-42
- 59.Arnett JJ. Emerging adulthood. A theory of development from the late teens through the twenties. Am Psychol. 2000 May;55(5):469–80.
- 60.Bramer WM, Giustini D, de Jonge GB, Holland L, Bekhuis T. De-duplication of database search results for systematic reviews in EndNote. J Med Libr Assoc. 2016 Jul;104(3):240–3. doi: 10.3163/1536-5050.104.3.014. https://europepmc.org/abstract/MED/27366130 .
- 61.Hoffmann TC, Glasziou PP, Boutron I, Milne R, Perera R, Moher D, Altman DG, Barbour V, Macdonald H, Johnston M, Lamb SE, Dixon-Woods M, McCulloch P, Wyatt JC, Chan AW, Michie S. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ. 2014 Mar 07;348:g1687. doi: 10.1136/bmj.g1687. https://core.ac.uk/reader/19913708?utm_source=linkout .bmj.g1687
- 62.Presseau J, Ivers NM, Newham JJ, Knittle K, Danko KJ, Grimshaw JM. Using a behaviour change techniques taxonomy to identify active ingredients within trials of implementation interventions for diabetes care. Implement Sci. 2015 Apr 23;10:55. doi: 10.1186/s13012-015-0248-7. https://implementationscience.biomedcentral.com/articles/10.1186/s13012-015-0248-7 .10.1186/s13012-015-0248-7
- 63.Fontaine G, Cossette S, Maheu-Cadotte MA, Deschênes MF, Rouleau G, Lavallée A, Pépin C, Ballard A, Chicoine G, Lapierre A, Lavoie P, Blondin J, Mailhot T. Effect of implementation interventions on nurses' behaviour in clinical practice: a systematic review, meta-analysis and meta-regression protocol. Syst Rev. 2019 Dec 05;8(1):305. doi: 10.1186/s13643-019-1227-x. https://systematicreviewsjournal.biomedcentral.com/articles/10.1186/s13643-019-1227-x .10.1186/s13643-019-1227-x
- 64.Fontaine G, Cossette S. A theory-based adaptive E-learning program aimed at increasing intentions to provide brief behavior change counseling: randomized controlled trial. Nurse Educ Today. 2021 Dec;107:105112. doi: 10.1016/j.nedt.2021.105112.S0260-6917(21)00369-5
- 65.Fontaine G, Cossette S. Development and design of E_MOTIV: a theory-based adaptive e-learning program to support nurses' provision of brief behavior change counseling. Comput Inform Nurs. 2023 Mar 01;41(3):130–41. doi: 10.1097/CIN.0000000000000942.00024665-202303000-00003
- 66.Sobell LC, Sobell MB. Timeline follow-back: a technique for assessing self-reported alcohol consumption. In: Litten RZ, Allen JP, editors. Measuring Alcohol Consumption. Totowa, NJ: Humana Press; 1992.
- 67.Stephens RS, Roffman RA, Simpson EE. Treating adult marijuana dependence: a test of the relapse prevention model. J Consult Clin Psychol. 1994;62(1):92–9. doi: 10.1037/0022-006x.62.1.92.
- 68.Harris RJ, Deeks JJ, Altman DG, Bradburn MJ, Harbord RM, Sterne JA. Metan: fixed- and random-effects meta-analysis. Stata J. 2008;8(1):3–28. doi: 10.1177/1536867x0800800102.
- 69.Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ. 2003 Sep 06;327(7414):557–60. doi: 10.1136/bmj.327.7414.557. https://europepmc.org/abstract/MED/12958120 .327/7414/557
- 70.Baumgartner C, Schaub MP, Wenger A, Malischnig D, Augsburger M, Walter M, Berger T, Stark L, Ebert DD, Keough MT, Haug S. CANreduce 2.0 adherence-focused guidance for internet self-help among cannabis users: three-arm randomized controlled trial. J Med Internet Res. 2021 Apr 30;23(4):e27463. doi: 10.2196/27463. https://boris.unibe.ch/id/eprint/164119 .v23i4e27463
- 71.Rooke S, Copeland J, Norberg M, Hine D, McCambridge J. Effectiveness of a self-guided web-based cannabis treatment program: randomized controlled trial. J Med Internet Res. 2013 Feb 15;15(2):e26. doi: 10.2196/jmir.2256. https://www.jmir.org/2013/2/e26/ v15i2e26
- 72.Tossmann HP, Jonas B, Tensil MD, Lang P, Strüber E. A controlled trial of an internet-based intervention program for cannabis users. Cyberpsychol Behav Soc Netw. 2011 Nov;14(11):673–9. doi: 10.1089/cyber.2010.0506.
- 73.StataCorp . StataCorp LLC. College Station, TX: StataCorp LLC; 2023. [2023-11-28]. Stata statistical software: release 18. https://www.stata.com/
- 74.Sterne JA, Savović J, Page MJ, Elbers RG, Blencowe NS, Boutron I, Cates CJ, Cheng HY, Corbett MS, Eldridge SM, Emberson JR, Hernán MA, Hopewell S, Hróbjartsson A, Junqueira DR, Jüni P, Kirkham JJ, Lasserson T, Li T, McAleenan A, Reeves BC, Shepperd S, Shrier I, Stewart LA, Tilling K, White IR, Whiting PF, Higgins JP. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ. 2019 Aug 28;366:l4898. doi: 10.1136/bmj.l4898. https://eprints.whiterose.ac.uk/150579/
- 75.McGuinness LA, Higgins JP. Risk-of-bias VISualization (robvis): an R package and Shiny web app for visualizing risk-of-bias assessments. Res Synth Methods. 2021 Jan;12(1):55–61. doi: 10.1002/jrsm.1411.
- 76.Haddaway NR, Page MJ, Pritchard CC, McGuinness LA. PRISMA2020: an R package and Shiny app for producing PRISMA 2020-compliant flow diagrams, with interactivity for optimised digital transparency and open synthesis. Campbell Syst Rev. 2022 Mar 27;18(2):e1230. doi: 10.1002/cl2.1230. https://europepmc.org/abstract/MED/36911350 .CL21230
- 77.Elliott JC, Carey KB. Correcting exaggerated marijuana use norms among college abstainers: a preliminary test of a preventive intervention. J Stud Alcohol Drugs. 2012 Nov;73(6):976–80. doi: 10.15288/jsad.2012.73.976.
- 78.Elliott JC, Carey KB, Vanable PA. A preliminary evaluation of a web-based intervention for college marijuana use. Psychol Addict Behav. 2014 Mar;28(1):288–93. doi: 10.1037/a0034995.2014-12693-008
- 79.Lee CM, Neighbors C, Kilmer JR, Larimer ME. A brief, web-based personalized feedback selective intervention for college student marijuana use: a randomized clinical trial. Psychol Addict Behav. 2010 Jun;24(2):265–73. doi: 10.1037/a0018859. https://europepmc.org/abstract/MED/20565152 .
- 80.Buckner JD, Zvolensky MJ, Lewis EM. On-line personalized feedback intervention for negative affect and cannabis: a pilot randomized controlled trial. Exp Clin Psychopharmacol. 2020 Apr;28(2):143–9. doi: 10.1037/pha0000304.2019-32623-001
- 81.Goodness TM, Palfai TP. Electronic screening and brief intervention to reduce cannabis use and consequences among graduate students presenting to a student health center: a pilot study. Addict Behav. 2020 Jul;106:106362. doi: 10.1016/j.addbeh.2020.106362.S0306-4603(19)31076-7
- 82.Palfai TP, Saitz R, Winter M, Brown TA, Kypri K, Goodness TM, O'Brien LM, Lu J. Web-based screening and brief intervention for student marijuana use in a university health center: pilot study to examine the implementation of eCHECKUP TO GO in different contexts. Addict Behav. 2014 Sep;39(9):1346–52. doi: 10.1016/j.addbeh.2014.04.025. https://europepmc.org/abstract/MED/24845164 .S0306-4603(14)00137-3
- 83.Riggs NR, Conner BT, Parnes JE, Prince MA, Shillington AM, George MW. Marijuana eCHECKUPTO GO: effects of a personalized feedback plus protective behavioral strategies intervention for heavy marijuana-using college students. Drug Alcohol Depend. 2018 Sep 01;190:13–9. doi: 10.1016/j.drugalcdep.2018.05.020.S0376-8716(18)30335-1
- 84.Bonar EE, Chapman L, Pagoto S, Tan CY, Duval ER, McAfee J, Collins RL, Walton MA. Social media interventions addressing physical activity among emerging adults who use cannabis: a pilot trial of feasibility and acceptability. Drug Alcohol Depend. 2023 Jan 01;242:109693. doi: 10.1016/j.drugalcdep.2022.109693.S0376-8716(22)00430-6
- 85.Bonar EE, Goldstick JE, Chapman L, Bauermeister JA, Young SD, McAfee J, Walton MA. A social media intervention for cannabis use among emerging adults: randomized controlled trial. Drug Alcohol Depend. 2022 Mar 01;232:109345. doi: 10.1016/j.drugalcdep.2022.109345. https://europepmc.org/abstract/MED/35144238 .S0376-8716(22)00082-5
- 86.Walukevich-Dienst K, Neighbors C, Buckner JD. Online personalized feedback intervention for cannabis-using college students reduces cannabis-related problems among women. Addict Behav. 2019 Nov;98:106040. doi: 10.1016/j.addbeh.2019.106040.S0306-4603(18)31438-2
- 87.Côté J, Tessier S, Gagnon H, April N, Rouleau G, Chagnon M. Efficacy of a web-based tailored intervention to reduce cannabis use among young people attending adult education centers in Quebec. Telemed J E Health. 2018 Nov;24(11):853–60. doi: 10.1089/tmj.2017.0144.
- 88.Cunningham JA, Schell C, Bertholet N, Wardell JD, Quilty LC, Agic B, Godinho A. Online personalized feedback intervention to reduce risky cannabis use. Randomized controlled trial. Internet Interv. 2021 Nov 14;26:100484. doi: 10.1016/j.invent.2021.100484. https://linkinghub.elsevier.com/retrieve/pii/S2214-7829(21)00124-X .S2214-7829(21)00124-X
- 89.Copeland J, Rooke S, Rodriquez D, Norberg MM, Gibson L. Comparison of brief versus extended personalised feedback in an online intervention for cannabis users: short-term findings of a randomised trial. J Subst Abuse Treat. 2017 May;76:43–8. doi: 10.1016/j.jsat.2017.01.009.S0740-5472(16)30294-X
- 90.Jonas B, Tensil MD, Tossmann P, Strüber E. Effects of treatment length and chat-based counseling in a web-based intervention for cannabis users: randomized factorial trial. J Med Internet Res. 2018 May 08;20(5):e166. doi: 10.2196/jmir.9579. https://www.jmir.org/2018/5/e166/ v20i5e166
- 91.Schaub MP, Wenger A, Berg O, Beck T, Stark L, Buehler E, Haug S. A web-based self-help intervention with and without chat counseling to reduce cannabis use in problematic cannabis users: three-arm randomized controlled trial. J Med Internet Res. 2015 Oct 13;17(10):e232. doi: 10.2196/jmir.4860. https://www.jmir.org/2015/10/e232/ v17i10e232
- 92.Sinadinovic K, Johansson M, Johansson AS, Lundqvist T, Lindner P, Hermansson U. Guided web-based treatment program for reducing cannabis use: a randomized controlled trial. Addict Sci Clin Pract. 2020 Feb 18;15(1):9. doi: 10.1186/s13722-020-00185-8. https://ascpjournal.biomedcentral.com/articles/10.1186/s13722-020-00185-8 .10.1186/s13722-020-00185-8
- 93.Kanfer FH. Implications of a self-regulation model of therapy for treatment of addictive behaviors. In: Miller WR, Heather N, editors. Treating Addictive Behaviors. Boston, MA: Springer; 1986. pp. 29–47.
- 94.Mohr DC, Cuijpers P, Lehman K. Supportive accountability: a model for providing human support to enhance adherence to eHealth interventions. J Med Internet Res. 2011 Mar 10;13(1):e30. doi: 10.2196/jmir.1602. https://www.jmir.org/2011/1/e30/ v13i1e30
- 95.Marlatt GA, Baer JS, Kivlahan DR, Dimeff LA, Larimer ME, Quigley LA, Somers JM, Williams E. Screening and brief intervention for high-risk college student drinkers: results from a 2-year follow-up assessment. J Consult Clin Psychol. 1998 Aug;66(4):604–15. doi: 10.1037/0022-006x.66.4.604.
- 96.Miller WR, Rollnick S. Motivational Interviewing: Preparing People for Change. New York, NY: Guilford Press; 2002.
- 97.Prince MA, Carey KB, Maisto SA. Protective behavioral strategies for reducing alcohol involvement: a review of the methodological issues. Addict Behav. 2013 Jul;38(7):2343–51. doi: 10.1016/j.addbeh.2013.03.010.S0306-4603(13)00082-8
- 98.Lundqvist TN. Cognitive Dysfunctions in Chronic Cannabis Users Observed During Treatment: An Integrative Approach. Stockholm, Sweden: Almqvist & Wiksell; 1997.
- 99.Kanter JW, Puspitasari AJ, Santos MM, Nagy GA. Behavioural activation: history, evidence and promise. Br J Psychiatry. 2012 May;200(5):361–3. doi: 10.1192/bjp.bp.111.103390.S0007125000271195
- 100.Jaffee WB, D'Zurilla TJ. Personality, problem solving, and adolescent substance use. Behav Ther. 2009 Mar;40(1):93–101. doi: 10.1016/j.beth.2008.03.001.S0005-7894(08)00062-2
- 101.Miller W, Rollnick S. Motivational Interviewing: Preparing People to Change Addictive Behavior. New York, NY: The Guilford Press; 1991.
- 102.Gordon JR, Marlatt GA. Relapse Prevention: Maintenance Strategies in the Treatment of Addictive Behaviors. 2nd edition. New York, NY: The Guilford Press; 2005.
- 103.Platt JJ, Husband SD. An overview of problem-solving and social skills approaches in substance abuse treatment. Psychotherapy (Chic) 1993;30(2):276–83. https://psycnet.apa.org/buy/1994-34928-001 .
- 104.Steinberg KL, Roffman R, Carroll K, McRee B, Babor T, Miller M. Brief counseling for marijuana dependence: a manual for treating adults. Center for Substance Abuse Treatment, Substance Abuse and Mental Health Services Administration, US Department of Health and Human Services. [2024-03-23]. https://store.samhsa.gov/product/brief-counseling-marijuana-dependence-manual-treating-adults/sma15-4211 .
- 105.de Shazer S, Dolan Y. More Than Miracles: The State of the Art of Solution-Focused Brief Therapy. Oxfordshire, UK: Routledge; 2007.
- 106.Copeland J, Swift W, Roffman R, Stephens R. A randomized controlled trial of brief cognitive-behavioral interventions for cannabis use disorder. J Subst Abuse Treat. 2001 Sep;21(2):55–65. doi: 10.1016/s0740-5472(01)00179-9.S0740-5472(01)00179-9
- 107.Linke S, McCambridge J, Khadjesari Z, Wallace P, Murray E. Development of a psychologically enhanced interactive online intervention for hazardous drinking. Alcohol Alcohol. 2008;43(6):669–74. doi: 10.1093/alcalc/agn066. https://europepmc.org/abstract/MED/18693217 .agn066
- 108.Wang ML, Waring ME, Jake-Schoffman DE, Oleski JL, Michaels Z, Goetz JM, Lemon SC, Ma Y, Pagoto SL. Clinic versus online social network-delivered lifestyle interventions: protocol for the get social noninferiority randomized controlled trial. JMIR Res Protoc. 2017 Dec 11;6(12):e243. doi: 10.2196/resprot.8068. https://www.researchprotocols.org/2017/12/e243/ v6i12e243
- 109.Sepah SC, Jiang L, Peters AL. Translating the diabetes prevention program into an online social network: validation against CDC standards. Diabetes Educ. 2014 Jul;40(4):435–43. doi: 10.1177/0145721714531339. https://escholarship.org/uc/item/1bw7s5z9 .0145721714531339
- 110.Cunningham JA, van Mierlo T. The check your cannabis screener: a new online personalized feedback tool. Open Med Inform J. 2009 May 07;3:27–31. doi: 10.2174/1874431100903010027. https://europepmc.org/abstract/MED/19587808 .
- 111.Bertholet N, Cunningham JA, Faouzi M, Gaume J, Gmel G, Burnand B, Daeppen JB. Internet-based brief intervention for young men with unhealthy alcohol use: a randomized controlled trial in a general population sample. Addiction. 2015 Nov;110(11):1735–43. doi: 10.1111/add.13051. https://serval.unil.ch/notice/serval:BIB_5651A3EE481E .
- 112.Walker DD, Roffman RA, Stephens RS, Wakana K, Berghuis J, Kim W. Motivational enhancement therapy for adolescent marijuana users: a preliminary randomized controlled trial. J Consult Clin Psychol. 2006 Jun;74(3):628–32. doi: 10.1037/0022-006X.74.3.628. https://europepmc.org/abstract/MED/16822119 .2006-08433-022
- 113.Miller MB, Leffingwell T, Claborn K, Meier E, Walters S, Neighbors C. Personalized feedback interventions for college alcohol misuse: an update of Walters and Neighbors (2005) Psychol Addict Behav. 2013 Dec;27(4):909–20. doi: 10.1037/a0031174. https://europepmc.org/abstract/MED/23276309 .2012-35005-001
- 114.Ajzen I. From intentions to actions: a theory of planned behavior. In: Kuhl J, Beckmann J, editors. Action Control. Berlin, Germany: Springer; 1985. pp. 11–39.
- 115.Dennis M, Titus JC, Diamond G, Donaldson J, Godley SH, Tims FM, Webb C, Kaminer Y, Babor T, Roebuck MC, Godley MD, Hamilton N, Liddle H, Scott CK. The Cannabis Youth Treatment (CYT) experiment: rationale, study design and analysis plans. Addiction. 2002 Dec 11;97 Suppl 1(s1):16–34. doi: 10.1046/j.1360-0443.97.s01.2.x.add03
- 116.Simons JS, Dvorak RD, Merrill JE, Read JP. Dimensions and severity of marijuana consequences: development and validation of the Marijuana Consequences Questionnaire (MACQ) Addict Behav. 2012 May;37(5):613–21. doi: 10.1016/j.addbeh.2012.01.008. https://europepmc.org/abstract/MED/22305645 .S0306-4603(12)00009-3
- 117.Legleye S. The Cannabis Abuse Screening Test and the DSM-5 in the general population: optimal thresholds and underlying common structure using multiple factor analysis. Int J Methods Psychiatr Res. 2018 Jun 10;27(2):e1597. doi: 10.1002/mpr.1597. https://europepmc.org/abstract/MED/29124816 .
- 118.Legleye S, Karila LM, Beck F, Reynaud M. Validation of the CAST, a general population Cannabis Abuse Screening Test. J Subst Use. 2009 Jul 12;12(4):233–42. doi: 10.1080/14659890701476532.
- 119.Sobell LC, Sobell MB. Timeline follow back. In: Litten RZ, Allen JP, editors. Measuring Alcohol Consumption: Psychosocial and Biochemical Methods. Totowa, NJ: Humana Press; 1992. pp. 41–72.
- 120.White HR, Labouvie EW, Papadaratsakis V. Changes in substance use during the transition to adulthood: a comparison of college students and their noncollege age peers. J Drug Issues. 2016 Aug 03;35(2):281–306. doi: 10.1177/002204260503500204.
- 121.White HR, Labouvie EW. Towards the assessment of adolescent problem drinking. J Stud Alcohol. 1989 Jan;50(1):30–7. doi: 10.15288/jsa.1989.50.30.
- 122.Cloutier RM, Natesan Batley P, Kearns NT, Knapp AA. A psychometric evaluation of the Marijuana Problems Index among college students: confirmatory factor analysis and measurement invariance by gender. Exp Clin Psychopharmacol. 2022 Dec;30(6):907–17. doi: 10.1037/pha0000531. https://europepmc.org/abstract/MED/34735206 .2022-00802-001
- 123.Guo H, Yang H, Yuan G, Zhu Z, Zhang K, Zhang X, Lu X, Zhang J, Du J, Shi H, Jin G, Hao J, Sun Y, Su P, Zhang Z. Effectiveness of information and communication technology (ICT) for addictive behaviors: an umbrella review of systematic reviews and meta-analysis of randomized controlled trials. Comput Hum Behav. 2023 Oct;147:107843. doi: 10.1016/j.chb.2023.107843.
- 124.Grigsby TJ, Lopez A, Albers L, Rogers CJ, Forster M. A scoping review of risk and protective factors for negative cannabis use consequences. Subst Abuse. 2023 Apr 07;17:11782218231166622. doi: 10.1177/11782218231166622. https://journals.sagepub.com/doi/10.1177/11782218231166622?url_ver=Z39.88-2003&rfr_id=ori:rid:crossref.org&rfr_dat=cr_pub0pubmed .10.1177_11782218231166622
- 125.Harkin B, Webb TL, Chang BP, Prestwich A, Conner M, Kellar I, Benn Y, Sheeran P. Does monitoring goal progress promote goal attainment? A meta-analysis of the experimental evidence. Psychol Bull. 2016 Feb;142(2):198–229. doi: 10.1037/bul0000025. https://eprints.whiterose.ac.uk/91437/ 2015-47216-001
- 126.Samdal GB, Eide GE, Barth T, Williams G, Meland E. Effective behaviour change techniques for physical activity and healthy eating in overweight and obese adults; systematic review and meta-regression analyses. Int J Behav Nutr Phys Act. 2017 Mar 28;14(1):42. doi: 10.1186/s12966-017-0494-y. https://ijbnpa.biomedcentral.com/articles/10.1186/s12966-017-0494-y .10.1186/s12966-017-0494-y
- 127.Garnett C, Crane D, Brown J, Kaner E, Beyer F, Muirhead C. Behavior Change Techniques Used in Digital Behavior Change Interventions to Reduce Excessive Alcohol Consumption: A Meta-regression. Ann Behav Med May 18. 2018;52(6):A. doi: 10.1093/abm/kax029.
- 128.Kaner EF, Beyer FR, Muirhead C, Campbell F, Pienaar ED, Bertholet N, Daeppen JB, Saunders JB, Burnand B. Effectiveness of brief alcohol interventions in primary care populations. Cochrane Database Syst Rev. 2018 Feb 24;2(2):CD004148. doi: 10.1002/14651858.CD004148.pub4. https://europepmc.org/abstract/MED/29476653 .
- 129.Sedrati H, Belrhiti Z, Nejjari C, Ghazal H. Evaluation of mobile health apps for non-medical cannabis use: a scoping review. Procedia Comput Sci. 2022;196:581–9. doi: 10.1016/j.procs.2021.12.052.
- 130.Curtis BL, Ashford RD, Magnuson KI, Ryan-Pettes SR. Comparison of smartphone ownership, social media use, and willingness to use digital interventions between generation Z and millennials in the treatment of substance use: cross-sectional questionnaire study. J Med Internet Res. 2019 Apr 17;21(4):e13050. doi: 10.2196/13050. https://www.jmir.org/2019/4/e13050/ v21i4e13050
- 131.Schünemann H, Brożek J, Guyatt G, Oxman A. The GRADE Handbook. London, UK: The Cochrane Collaboration; 2013.
- 132.Guyatt GH, Oxman AD, Schünemann HJ, Tugwell P, Knottnerus A. GRADE guidelines: a new series of articles in the Journal of Clinical Epidemiology. J Clin Epidemiol. 2011 Apr;64(4):380–2. doi: 10.1016/j.jclinepi.2010.09.011.S0895-4356(10)00329-X
- 133.Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, Schünemann HJ. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008 Apr 26;336(7650):924–6. doi: 10.1136/bmj.39489.470347.AD. https://europepmc.org/abstract/MED/18436948 .336/7650/924
- 134.Hjorthøj CR, Hjorthøj AR, Nordentoft M. Validity of Timeline Follow-Back for self-reported use of cannabis and other illicit substances--systematic review and meta-analysis. Addict Behav. 2012 Mar;37(3):225–33. doi: 10.1016/j.addbeh.2011.11.025.S0306-4603(11)00387-X
- 135.Robinson SM, Sobell LC, Sobell MB, Leo GI. Reliability of the Timeline Followback for cocaine, cannabis, and cigarette use. Psychol Addict Behav. 2014 Mar;28(1):154–62. doi: 10.1037/a0030992.2012-34893-001
- 136.Abraham C, Michie S. A taxonomy of behavior change techniques used in interventions. Health Psychol. 2008 May;27(3):379–87. doi: 10.1037/0278-6133.27.3.379.2008-08834-010
- 137.Garrett JJ. The Elements of User Experience: User-Centered Design for the Web and Beyond. London, UK: Pearson Education; 2010.
- 138.Lorencatto F, West R, Bruguera C, Brose LS, Michie S. Assessing the quality of goal setting in behavioural support for smoking cessation and its association with outcomes. Ann Behav Med. 2016 Apr 24;50(2):310–8. doi: 10.1007/s12160-015-9755-7. https://europepmc.org/abstract/MED/26603626 .10.1007/s12160-015-9755-7
Associated Data
Supplementary Materials
Data Availability Statement
The data sets generated and analyzed during this study are available from the corresponding author on reasonable request.