Therapist effects in real-world rehabilitation outcomes: a cohort study of the nationwide GLA:D osteoarthritis management program in Denmark
1Department of Liberal Studies, College of Administrative and Business Studies, Niger State Polytechnic, Bida Campus, Nigeria
2Centre for Applied Health & Social Care Research (CARe), Sheffield Hallam University, UK
3School of Health & Social Care, Sheffield Hallam University, UK
4School of Medicine, Keele University, UK
5Center for Muscle and Joint Health, Department of Sports Science and Clinical Biomechanics, University of Southern Denmark, Odense, Denmark
6The Research and Implementation Unit PROgrez, Department of Physiotherapy and Occupational Therapy, Central and West Zealand Hospital, Denmark
7Department of Research, Central and West Zealand Hospital, Denmark
✉Corresponding author: Professor George M. Peat, Centre for Applied Health and Social Care Research (CARe), School of Health and Social Care, College of Health, Wellbeing and Life Sciences, Sheffield Hallam University, Robert Winston Building, Collegiate Campus, Broomhall Road, Sheffield, S10 2BP, United Kingdom. E: g.peat@shu.ac.ukABSTRACT
Objective
Unlike several other fields of healthcare, little is known about the size of ‘therapist effects’ on patient outcomes following rehabilitation for musculoskeletal conditions. We aimed to estimate the proportion of variance in patient outcomes from a structured rehabilitation program explained by therapist effects.
Methods
For our observational cohort study we accessed data from the national multicentre Good Life with osteoArthritis in Denmark (GLA:D) osteoarthritis management program. Analyses included 23,021 consecutive eligible adults with hip or knee osteoarthritis (mean (SD) age 65.0 (9.8) years, 71% female) treated by 657 therapists between October 2014 and February 2019. The primary outcome was ≥30% reduction in pain intensity on 0-100 VAS at 3 months. Therapist effects were estimated as the variance partition coefficient (intra-class correlation coefficient (ICC)) from two-level random intercept logistic regression models before and after adjusting for patient-level case-mix factors and therapist-level characteristics (number of patients treated, days since therapist certification). Analyses were repeated for a range of secondary outcomes using multiply imputed data and complete-case analysis.
Results
52% of patients reported a ≥30% reduction in pain intensity on 0-100 VAS at 3 months. In the null model the ICC was 0.007 (95%CI: 0.005, 0.009), which changed little after adjusting for patient- and therapist-level covariates. Upper confidence limits for ICC estimates across all secondary outcomes in multiply imputed and complete case analyses were less than 0.03.
Conclusions
In a nationally implemented osteoarthritis management program delivered by trained healthcare professionals, therapist effects made a minimal contribution to variation in patient outcomes.
Article notes
Competing Interest Statement
EMR is the copyright holder of Knee injury and Osteoarthritis Outcome Score (KOOS) and several other patient-reported outcome measures, and co-founder of the Good Life with Osteoarthritis in Denmark (GLA:D), a not-for profit initiative to implement clinical guidelines in primary care hosted by University of Southern Denmark. STS has received personal fees from Munksgaard and TrustMe-Ed, outside the submitted work, and is co-founder of GLA:D. PEO, DA, JP, DY, DTG, GMP have no competing interests to declare.
Funding Statement
This study was funded by the Danish Physiotherapy Association, the Danish Rheumatism Association, the Physiotherapy Practice Foundation, Region Zealand (Exercise First), EU Horizon 2020 (No 945377 (ESCAPE)), Innovative Health Initiative Joint Undertaking (IHI JU) (No. 101219324 (PROBE)), Danish Regions (No R232-A5132), The Department of Sports Science and Clinical Biomechanics, Faculty of Health Sciences, University of Southern Denmark, NSR Hospitals, Denmark (No A1683), School of Health and Social Care, Sheffield Hallam University, and the EPSRC South Yorkshire Digital Health Hub (EP/X03075X/1).
INTRODUCTION
The range, difficulty, and variability of behaviours required of health professionals are recognised as key mechanisms through which many complex healthcare interventions produce patient benefit.[1] Therefore, studies that set out to estimate the magnitude of “therapist effects” - defined as the effect of a given therapist on patient outcomes as compared to another therapist[2] - have the potential to drive important new insights and lead to improvements in care: Johns et al. (2019)[3] list four potential ways that studies of therapist effects contribute to knowledge and care: they temper an over-emphasis on ‘brands’ of treatment; they identify more or less effective therapists with implications for matching patients, as well as selection, training, and revalidation; they contribute to mechanistic understanding; and they generate research questions designed to reduce unwarranted variability in service provision and outcomes.
In psychotherapy, surgery, and medicine, where there is a relatively long history of such studies, there is strong evidence that therapist effects contribute to patient outcomes, although estimates vary substantially. Whilst extreme estimates suggest that between 0 and 47% of the variation in patient outcomes may be attributable to therapist effects, the more plausible range consistently observed is 3-10%.[3–7] Factors such as the volume of procedures undertaken by individual surgeons[8] or psychotherapists’ interpersonal skills[9] have been proposed as potential determinants.
A comparable body of evidence is largely absent for musculoskeletal rehabilitative interventions. We found only five studies, conducted over the last 15 years, which have estimated therapist effects in this field (Table 1).[10–14]
None met the suggested requirement for multi-level models of therapist effects that studies include outcomes from at least 100 healthcare professionals treating at least 10 patients each.[3] One reason for the apparent paucity of evidence in musculoskeletal rehabilitation has been a lack of available large-scale registry data that combine the collection of standardised patient outcome measures and unique identifiers for therapists. The national rollout of structured osteoarthritis management programs with accompanying registry data created the opportunity to examine therapist effects in this field. The nationwide Good Life with osteoArthritis in Denmark (GLA:D®) program is an 8-week structured, group-based, physiotherapist-led osteoarthritis management program for people with hip or knee osteoarthritis comprising 2-3 patient education sessions and 12 clinician-supervised exercise therapy sessions. Since beginning in 2013, tens of thousands of patients have taken part in programs delivered by trained physiotherapists across Denmark, mainly in primary care centres and municipal settings.[15]
Given the unique opportunity afforded by the de-identified data from the GLA:D registry, our aim was to estimate ‘real world’ therapist effects on patient-reported outcomes from a structured, supervised group-based musculoskeletal rehabilitation intervention.
METHODS
Data source
The Danish national, electronic GLA:D® registry houses data on participant characteristics and outcomes collected at baseline, three months, and 12 months via a combination of patient-reported, therapist-reported, and objective measures, and the routine collection of standard outcomes is an integral component of the GLA:D® program. The GLA:D® registry was approved by the Danish Data Protection Agency. According to the Danish Data Protection Act, patient consent is not required as personal data was processed exclusively for research and statistical purposes. Separate ethics approval was not needed for the current analysis.
Population
For the current analysis all consecutive participants with hip or knee OA enrolled on the GLA:D® program in Denmark between 9 October 2014 and 28 February 2019 were potentially eligible. These dates represent a period before COVID-19 during which the outcome measures, exposures, and covariates of interest in this analysis were included in the data-collection instruments. Participants who had not returned a patient-reported questionnaire at baseline or who did not have a completed therapist ID were excluded from our analyses. We included only participants with a baseline pain intensity score ≥40 out of 100, a common eligibility criterion for clinical trials in osteoarthritis.[16,17] For participants taking the program more than once, only the first (index) attendance was included in the analysis.
Outcomes
The primary outcome of interest was clinically important pain reduction at 3 months, defined as ≥30% reduction in pain intensity (0-100 VAS) between baseline and 3 months.[17,18] Secondary outcomes were ≥50% reduction in pain intensity (0-100 VAS), pain intensity score (0-100 VAS) at 3 months, HOOS/KOOS Quality of Life subscale score (0-100) at 3 months, EQ5D Health Utility Score (5L Danish values, -0.757 – 1.000) at 3 months, and EQ5D Health-Related Quality of Life VAS (0-100) at 3 months. Pain and quality of life (hip/knee-related and general health-related) represent two of the five domains for core outcome sets in hip and knee osteoarthritis trials (the others being physical function, patient global assessment, and adverse events),[19,20] and are among highly-rated optional recommended domains for evaluating osteoarthritis management programs[21]
Therapists
All certified healthcare practitioners delivering the GLA:D intervention complete 2-day standardised training. The majority are physiotherapists. Analyses were restricted to therapists who had treated at least 10 patients.[3] Detailed information on therapists is not routinely collected. However, we used cumulative number of patients treated and number of days since completing GLA:D training as indicators of therapist experience.
Patient-level covariates for case-mix adjustment
Differences in case-mix may explain apparent therapist effects. For case-mix adjustment in our analyses we identified the following potential patient-level determinants of pain outcome from previous literature[22] and previous analyses of GLA:D data[23–26]: patient age, sex, born in Denmark, Danish citizen, month and year of entry to GLA:D program, most affected joint (hip/knee), body mass index (kg/m2), duration of symptoms, walking speed time during 40 metre walk test at baseline (m/s), number of chairs stands completed in 30 seconds at baseline, number of painful body areas at baseline (0-56), UCLA physical activity score at baseline (1-10), EQ-5D VAS at baseline (0-100), EQ-5D Health Utility score (5L Danish values - 0.757 - 1.000), KOOS/HOOS Quality of Life score at baseline (0-100), Arthritis Self-Efficacy Scale Pain and Other subscale scores (10-100), SF-12 Physical and Mental Health Component Scores (0-100), pain intensity (0-100) at baseline.
Statistical analysis
The rate and pattern of missing data were evaluated using the naniar package in R. Rates of missing data ranged from 0% to 30% (HOOS/KOOS QOL score at 3 months) due mainly to loss to follow-up (27%) and removal/later inclusion of symptom duration and SF-12 baseline between 2014 and 2019 (i.e. missing by design) (Supplementary data). Our primary analysis was of multiply imputed data, judging that complete case analysis or simple imputation were insufficient due to the risk of bias.[27] Imputation models were based on the assumption of data missing at random (MAR). Given the concentration of missingness in a few variables, we analysed patterns and probabilities of missing data to identify predictors for missing values.[28,29] The imputation model included all variables in the final model, including outcome and auxiliary variables (secondary outcomes).[30] Missing data were imputed using multiple imputation with chained equations using the mice package in R to create 40 imputed datasets, to equal or exceed the fraction of missing data[31] and using 10 iterations each. To achieve model convergence, continuous predictors were standardised and centred.
For binary outcomes, we fitted two-level (patients at level 1 nested within therapists at level 2) random intercept logistic regression models (Table 2). Three models were fitted. Model 0representes the ‘null’ model with no covariates. Model 1 includes the patient-level (level-1) covariates. Model 2 is the full model consisting of Model 1 and the therapist-level (level-2) covariates. In each model we reported the Variance Partition Coefficient (VPC) estimated by the intra-class correlation coefficient (ICC) using the latent variable method, together with model fit statistics (Akaike Information Criterion (AIC), Bayesian Information Criterion (BIC), and area under the Receiver Operator Curve (AUC)). For continuous secondary outcomes, random-intercept linear regression models were fitted in the same sequence and using AIC and BIC to evaluate model fit.
To visually display variation between therapists in patient outcomes we produced funnel plots of the case-mix-adjusted outcomes (therapist-specific observed - expected outcome + overall mean outcome) by number of patients seen per therapist. 95% and 99.8% confidence limits were corrected for an overdispersion factor (phi), calculated using observed variance/expected variance.[32,33]
We compared the above models and outputs from multiply imputed datasets with the same analyses conducted on a complete case dataset of therapists with at least 10 patients, each with fully observed data on all predictors and outcomes.
As a final step, we conducted an exploratory descriptive comparison of patient outcomes for selected anonymised therapists who were identified as potential outliers from the above funnel plots.
Analyses were conducted in R4.4.0. A list of packages used in R is provided in Supplementary data.
Patient and public involvement
Patients and members of the public were not involved in the conceptualisation, design, analysis, interpretation, or dissemination of this study.
RESULTS
Table 3 provides the descriptive characteristics of 657 therapists and 23,021 patients included in the current analysis. The median number of patients per therapist was 26 (IQR 16, 44) and therapists were a median of 640 days post-certification (IQR 305, 1073). Patients had a mean (SD) age of 65.0 (9.8) years, 71% were female, and they had typically attended higher (post-secondary) education (59%), and most were either retired (53%) or employed/student (29%). The knee joint was the primary complaint in 74% of patients.
Pain intensity (0-100 VAS) at baseline was typically moderate to severe (mean 61.3, SD 14.5). Of 16,746 participants with pain intensity scores available at baseline and 3 months, 8,759 (53%) had experienced a 30% or greater reduction in pain intensity at 3 months; 5,750 (34%) experienced a 50% or greater reduction in pain intensity.
In the variance partition coefficient model for the primary outcome in multiply imputed data, the estimated intra-class correlation coefficient was 0.007 (0.005, 0.009) (Table 4).
Baseline patient characteristics associated (p<0.05) with increased odds of achieving a ≥30% reduction in pain at 3 months were: younger age, female sex, primary complaint was knee problem, lower BMI, shorter duration of complaint, faster walking speed, greater number of chair stands, fewer areas of body pain, higher arthritis self-efficacy scores, higher HOOS/KOOS Quality of Life subscale scores, higher EQ5D VAS scores, lower SF-12 Physical Component Scale scores, higher SF-12 Mental Component Scale scores, and higher pain intensity. Although the AIC remained constant across the three models, the BIC for model 2 was lower than that for model 1, indicating that the addition of therapist-level characteristics did improve model fit. However, therapist-level predictors - cumulative number of patients treated (aOR 0.97 (95%CI: 0.93, 1.02)) and number of days since completing GLA:D training (0.99 (0.94, 1.04)) – were not statistically significantly associated with the outcome. The ICC (0.007) in the null model was statistically significant and effectively unchanged after adjusting for patient-level characteristics (0.008) and therapist-level characteristics (0.008).The AUC of 0.66 suggests weak discriminative ability.
The funnel plot showing therapist-specific outcomes adjusted for patient- and therapist-level covariates and corrected for overdispersion, identified two outlier therapists with better than expected outcomes at the 99.8% confidence threshold (Figure 1).
Across all secondary outcomes and models in the multiply imputed data analyses, the pattern of predictor-outcome associations and the magnitude of the ICC was similar (Table 5; Supplementary Data).
Complete case analysis included 9720 participants and 403 therapists with complete data on all outcomes and covariates. Across all outcomes and models, the ICC estimates were systematically higher than in the analyses of multiply imputed data. However, the upper 95% confidence limits for all ICC estimates was below 0.03 (Table 5; Supplementary Data).
DISCUSSION
Summary of key findings
The present study used data available from a large-scale national registry to estimate the magnitude of ‘therapist effects’ on short-term patient-reported pain and quality of life outcomes for adults enrolled on a structured, group-based, physiotherapist-led osteoarthritis management program. Our multi-level models suggested small or very small therapist effects, accounting for less than 3% of the variance across all outcomes, analyses, and models.
Comparison with previous studies
Our estimates are at the lowest end of the range of 1-12% reported in previous studies of therapist effects in musculoskeletal pain conditions,[10–14] and the 3-12% range typically observed in other conditions and settings[4–7,34]. Before speculating on substantive reasons why this might be, it is important to consider whether this could reflect model mis-specification and unreasonable assumptions during modelling.[35]
Excluding outlier therapists would be expected to reduce therapist effects. Selection of therapists for inclusion in our analyses was based only on having seen a minimum of 10 patients – a suggested minimum for such analyses.[3] We were careful not to select or exclude any therapists based on their characteristics or outcomes, including the number of their patients providing incomplete data and missing outcome data at 3 months. Overall, 27% of eligible participants did not have the primary outcome observed, which was expected to vary by therapist, and could not be assumed to be missing completely at random. In these circumstances complete-case analysis not only loses statistical power but is likely to be susceptible to bias.[36] Our primary analysis was therefore based on multi-level multiply imputed data, including imputing missing outcome data. Our findings assume data missing at random and correct specification of the multiple imputation procedures. We did not find substantially larger estimates from complete case analysis, suggesting that any mis-specification of the imputation procedures may be unlikely to mask substantially higher ICC values. Nevertheless, it is possible that the (self-)selection and training of therapists to deliver a standardised intervention effectively reduces the amount of therapist variance that is present in the data. Indeed, this would be an intentional goal to ensure the GLA:D program is delivered in a consistent manner. In studies of psychotherapy, manualised treatment has been associated with lower therapist effects.[37] Previous studies in musculoskeletal rehabilitation using routine or registry data were largely based on unselected physiotherapists providing an unspecified mixture of treatment approaches under ‘usual care’.[11–14] Lewis et al.[10] provide direct comparisons of ICC values between trial treatment arms, including usual care, but the highly variable ICC estimates likely also reflected limited sample sizes for their stratified analyses.
Inappropriate model assumptions and analytic processes may also inflate patient variation, and consequently reduce the contribution of therapist effects to total variation in outcomes. The choice of outcome measure and timing of endpoint seem unlikely to explain the difference: previous studies have used a mixture of pain/symptom and functional outcomes and endpoints with no obvious relationship to ICC estimates emerging. ICC estimates from two trials using self-reported functional outcome at 3 months were 2.6%[38] and 2.7%[39] respectively. Our study chose a combination of binary and continuous outcomes across three domains, pain, hip/knee-specific quality of life, and health-related quality of life. The magnitude of ICC estimates did not differ substantially from one outcome to another.
Strengths and Limitations
This the largest study to date of therapist effects in musculoskeletal rehabilitation, made possible by the routine collection and recording of consistent data on patient-reported outcomes and therapists at scale in the nationwide GLA:D program. It comfortably exceeds the suggested guidance of a minimum of 100 therapists each treating at least 10 patients. We were able to incorporate a large number of patient-level covariates in the case-mix adjustment. Higher pain severity, longer symptom duration, multiple site pain, older age, and higher BMI were consistently associated with worse outcomes in the current study. These are well-recognised prognostic indicators of poor outcome in musculoskeletal pain conditions.[40]
There were some limitations in the data available. “Ability to participate in daily activities” is a core outcome domain for osteoarthritis management programs[21] but KOOS-12 Function was introduced only partway through the period covered in the current analysis. Few therapist-level factors were available within the GLA:D registry data, and this meant it was not possible to extend previous work exploring the importance of personality traits and attitudes for therapist effects.[11–14] Outcome measures were available at 12 months, however we reasoned that therapist effects would most likely be observed in the short-term. An alternative hypothesis might be that some therapists are more effective at engaging patients and creating sustained outcomes. Future studies of longer-term outcomes could investigate this. Similarly, it could be of interest to extend our analyses to other outcomes such as functional performance tests. Our choice of 30% reduction in pain intensity for primary analysis is open to challenge. OMERACT-OARSI criteria[41], for example, consider 20% improvement (coupled with improvement in function or patient global assessment) as clinically important improvement. However, our secondary analyses included a higher threshold (50%) and pain intensity as a continuous outcome, and found consistent ICC estimates. Based on this, we would not expect a change to 20% threshold to alter our findings on therapist effects.
GLA:D is delivered as a group intervention, whereas the previous estimates of therapist effects in musculoskeletal rehabilitation have come from individual treatment. If some of the variation in patient outcomes is due to interactions between group members independent of the therapist then this may contribute to the lower estimates of therapist effects in the current study.
Implications for research and/or practice
Implications of our findings depend to some extent on whether therapist effects are viewed as contextual or specific effects, and whether we believe that selection and training of therapists for GLA:D, together with standardisation of the GLA:D intervention, have effectively minimised the scope for variation in therapist effects on patient outcomes. Therapist effects may be viewed as part of wider concepts of contextual effects, often additive to ‘specific’ treatment effects.[42] Substantial contextual factors are seen across most treatments for osteoarthritis[43,44], and seem likely also for exercise-based interventions.[45] Our findings extend this work by suggesting that contextual factors that relate to therapist effects – therapist characteristics or therapist-patient interaction and alliance[46] - make a minimal contribution to variation in patient outcomes from this structured, group-based rehabilitation intervention. Any contextual effects must be attributable to alternative sources, e.g. patient expectations, intervention setting.
Therapist effects may also be viewed as integral to ‘specific’ treatment effects.[47,48] The recently developed core capability framework for qualified healthcare professionals asserts that health professionals require a diverse array of skills to provide optimal care, including rehabilitative interventions, for all people with OA.[49]
The lack of therapist effects seen in our study could be consistent with two alternative explanations: either these capabilities are fairly consistently present among all therapists delivering the GLA:D intervention, or that patient outcomes from this structured program are not particularly dependent on them. Determining between these competing explanations has important implications for the selection and training of therapists to deliver structured rehabilitation programs, perhaps even the expansion of this role to other personnel. The emergence of osteoarthritis management programs internationally, with varying structure, context, and content, may present opportunities not just for replication of our findings but crucial insights that discriminate between these competing alternative explanations[50]. At the present time, our study suggests that further efforts to select and match therapists to deliver osteoarthritis management programs, or to monitor therapist-level outcomes over and above current practice in GLA:D appear unwarranted.
Conclusion
National GLA:D registry data provide a unique opportunity to investigate the magnitude of therapist effects on patient outcomes from a structured, group-based osteoarthritis management program. We found that therapist effects – whether viewed as contextual factors or integral to specific treatment effects - made a minimal contribution to variation in patient outcomes.
Supporting information
Data Availability
The data used in this study cannot be shared publicly because of potentially identifiable or sensitive information. Data may be accessed upon reasonable request by contacting the GLA:D administration.
Funding
The initiation of the Good Life With Osteoarthritis in Denmark Program was partly funded by the Danish Physiotherapy Association’s fund for research, education, and practice development; the Danish Rheumatism Association; and the Physiotherapy Practice Foundation. STSs research is currently funded by a program grant from Region Zealand (Exercise First), a consortium grant from the European Union’s Horizon 2020 research and innovation program under grant agreement No 945377 (ESCAPE) and an Innovative Health Initiative Joint Undertaking (IHI JU) under grant agreement No. 101219324 (PROBE). DTG is currently funded by a grant from Danish Regions (No R232-A5132, a faculty grant from The Department of Sports Science and Clinical Biomechanics, Faculty of Health Sciences, University of Southern Denmark, and a grant from NSR Hospitals, Denmark (No A1683), which all are outside the submitted work. Data access costs were funded by an award from the School of Health and Social Care, Sheffield Hallam University. GMP is part funded by the EPSRC South Yorkshire Digital Health Hub (EP/X03075X/1).
Ethical approval
The GLA:D® registry was approved by the Danish Data Protection Agency. According to the Danish Data Protection Act patient consent is not required as personal data was processed exclusively for research and statistical purposes. Separate ethics approval was not needed for the current analysis.
Contributorship
PEO: formal analysis, methodology, software, visualization, writing – original draft, writing – review and editing; JP: conceptualization, methodology, writing – original draft, writing – review and editing; DA: conceptualization, methodology, writing – original draft, writing – review and editing; DY: conceptualisation, methodology, writing – review and editing; DTG: Data curation, investigation, methodology, project management, resources, writing – review and editing; EMR: Funding acquisition, investigation, methodology, supervision, writing – review and editing; STS: Funding acquisition, investigation, methodology, supervision, writing – review and editing; GMP: conceptualisation, formal analysis, funding acquisition, methodology, project management, software, supervision, validation, visualization, writing – original draft, writing – review and editing
Competing interests
EMR is the copyright holder of Knee injury and Osteoarthritis Outcome Score (KOOS) and several other patient-reported outcome measures, and co-founder of the Good Life with Osteoarthritis in Denmark (GLA:D®), a not-for profit initiative to implement clinical guidelines in primary care hosted by University of Southern Denmark. STS has received personal fees from Munksgaard and TrustMe-Ed, outside the submitted work, and is co-founder of GLA:D®. PEO, DA, JP, DY, DTG, GMP have no competing interests to declare.
Data Sharing
The data used in this study cannot be shared publicly because of potentially identifiable or sensitive information. Data may be accessed upon reasonable request by contacting the GLA:D® administration.
Acknowledgements
The authors gratefully acknowledge the contribution of Professor Khaled Khatab to the design, analysis, and interpretation of the work. For the purpose of open access, the author has applied a Creative Commons Attribution (CC BY) licence to any Author Accepted Manuscript version arising from this submission.