Digital behavioural tests as diagnostic aid for psychosis
1Translational Research Exchange @ Exeter, Living Systems Institute, Department of Mathematics and Statistics, Faculty of Environment, Science and Economy, University of Exeter, UK
2Department of Psychology, Faculty of Health and Life Sciences, University of Exeter, UK
3Research & Development Department, Devon Partnership NHS Trust, Exeter, UK
4Faculty of Health Sciences, Bristol Medical School, University of Bristol, UK
5Specialist Team for Early Psychosis, Devon Partnership NHS Trust, Exeter, UK
6EPSRC Hub for Quantitative Modelling in Healthcare University of Exeter, Exeter, EX4 4QJ, UK
*Corresponding author: Piotr Słowiński, p.m.slowinski@exeter.ac.ukAbstract
Timely interventions have a proven benefit for people experiencing psychotic illness. One bottleneck to accessing timely interventions is the referral process to the specialist team for early psychosis (STEP). Many general practitioners lack awareness or confidence in recognising psychotic symptoms or state. Additionally, referrals for people without apparent psychotic symptoms, although beneficial at a population level, lead to excessive workload for STEPs. There is a clear unmet need for accurate stratification of STEPs users and healthy cohorts. Here we propose a new approach to addressing this need via the application of digital behavioural tests.
To discriminate between the STEPs users (SU; n=32) and controls (n=32, age and sex matched), we employed k-nearest neighbours (kNN) classifier, and applied it to objective, quantitative and interpretable features derived from the ‘mirror game’ (MG) and trail making task (TMT). The MG is a movement coordination task shown to be a potential socio-motor biomarker of schizophrenia, while TMT is a neuropsychiatric test of cognitive function. We show that the proposed classifier achieves an excellent performance, AUC = 0.89 (95%CI 0.73-1), Sensitivity = 0.75 (95%CI 0.5-1), Specificity = 1 (95%CI 0.62-1), evaluated on 25% hold-out and 1000 folds. We demonstrate that this performance is underpinned by the large effect sizes of the differences between the cohorts in terms of the features used for classification. We also find that MG and TMT are unsuitable in isolation to successfully differentiate between SU with and without at-risk-mental-state or first episode psychosis with sufficient level of performance.
Our findings show that introduction of standardised battery of digital behavioural tests could benefit both clinical and research practice. Including digital behavioural tests into healthcare practice could allow precise phenotyping and stratification of the highly heterogenous population of people referred to STEPs resulting in quicker and more personalised diagnosis. Moreover, the high specificity of digital behavioural tests could facilitate the identification of more homogeneous clinical high-risk populations, benefiting research on prognostic instruments for psychosis. In summary, our study demonstrates that cheap off-the-shelf equipment (laptop computer and a leap motion sensor) can be used to record clinically relevant behavioural data that could be utilised in digital mental health applications.
Funding
EPSRC Impact Acceleration Account, Impact & Knowledge Exchange Award, Jean Golding Institute seed corn, Avon & Wiltshire Mental Health Partnership NHS Trust Research Capability Funding.
PS was generously supported by the Wellcome Trust Institutional Strategic Support Award 204909/Z/16/Z. KTA gratefully acknowledges the financial support of the EPSRC via grant EP/T017856/1.
For the purpose of open access, the authors have applied a ‘Creative Commons Attribution (CC BY) licence to any Author Accepted Manuscript version arising.
Article notes
Competing Interest Statement
The authors have declared no competing interest.
Clinical Protocols
Funding Statement
This study was funded by: EPSRC Impact Acceleration Account, Impact & Knowledge Exchange Award, Jean Golding Institute seed corn, Avon & Wiltshire Mental Health Partnership NHS Trust Research Capability Funding. PS was generously supported by the Wellcome Trust Institutional Strategic Support Award 204909/Z/16/Z. KTA gratefully acknowledges the financial support of the EPSRC via grant EP/T017856/1.
Summary of Updates:
INTRODUCTION
Psychosis is a severe mental illness characterised by loss of contact with reality and symptoms such as hallucinations, delusions and thought disorders. It can be one of the first symptoms of a range of serious and long-term mental disorders such as schizophrenia, affective and other psychoses. Developing psychosis in young adulthood is devastating and often disrupts the trajectory into healthy and independent adulthood; the mean age of onset is 22 years for men and 26 years for women.1 People with serious mental illness die 15 to 20 years earlier than the general population.2
Serious mental disorders are extremely expensive to treat, with presence of psychotic or affective symptoms being one of patient characteristics driving increase in hospital costs.3 Their direct healthcare toll to NHS England have been estimated at £2.82 billion annually in 2019.3 While the most recent estimate puts the overall annual economic impact of schizophrenia and psychosis in England at £11.8 billion.4 The higher overall economic impact includes reduced labour supply, premature mortality, reduced health-related quality of life, lost output, lost tax revenue, transfer payments, and unpaid care by family or friends.
Most risk factors for a poor outcome, such as gender or low socio-economic status, are difficult or impossible to alter. But people with psychosis have better outcomes if they are treated as soon as possible after their first symptom.5 Early interventions can reduce the rate of relapse, risk of suicide, and number of hospital admissions.1,6 They also significantly improve quality of life by enabling people to finish education and develop supportive networks outside the family of origin.1
Early interventions are typically delivered by a specialist team for early psychosis (STEP).7–12 However the referral process to STEPs is far from being optimal. Although most STEP referrals are from primary care, many general practitioners lack awareness of high-risk symptoms or are not confident with recognising the psychotic state, both of which could lead to young people not receiving the care they need.13,14 On the other hand, although increasing the number of referrals has been shown to be beneficial at the population level, it also leads to increases in STEPs workload thus contributing to the pressure on the care system. The extra work is caused by higher number of assessments requested as well as a need for an increased engagement (including dedicated liaison practitioners) with primary care providers to identify and refer people experiencing, or at risk of, psychotic illness.9,15
Here, we investigate if digital behavioural tests can be used as an effective tool that allows differentiation between people referred to STEPs and the general population, and if they show potential to facilitate and standardise the referral process. Specifically, we use data from a digital version of the trail making task (TMT), a standard method for assessment of cognitive function,16,17 and the mirror game (MG), a novel way of assessing socio-motor functioning (motor coordination and interpersonal synchronisation).18,19 Our choice of the tasks is based on a significant body of research showing that assessment of movement, behaviour and cognitive function allows to accurately differentiate between people with schizophrenia and general population.18,20–31 In particular, motor and executive functions18,30 as well as eye movements24,27 were shown to hold promising diagnostic potential. In addition, deficits in motor coordination were recently shown to be markers of long-term clinical outcomes,31 while performance in TMT was shown to differ in people at clinical high-risk for psychosis who transitioned from those who did not transition to psychosis.32 As part of our study we also test feasibility of using cheap off-the-shelf components (laptop computer with a plug-in sensor) for simplifying neuropsychiatric assessment and introducing standardised digital tests to clinical practice.20
METHODS
Study design and participants
The study was designed as a prospective, cross-sectional feasibility study in a group of service users accepted for an assessment for psychosis including people with first episode psychosis or assessed as being at risk of developing psychosis. Control cohort was recruited independently at the University of Exeter (UoE). Demographic and clinical characteristics of participants can be found in Table 1.
Service users (SU) were identified and recruited by Devon Partnership NHS Trust (DPT) and Avon and Wiltshire Mental Health Partnership NHS Trust (AWP). In total we recruited 32 participants, all of which were included in the analysis (we do not have data about number of screened participants, none of the participants dropped-out). The inclusion criteria were being accepted for an assessment for psychosis or risk of developing psychosis by a consultant psychiatrist or a trained specialist with experience in at-risk mental states. The exclusion criteria were:
- Lacking capacity to provide informed consent for inclusion. The clinical team had an opportunity to assess the mental capacity at the CAARMS (comprehensive assessment of at-risk mental state (ARMS)33 appointment before potential participants were approached to request consent to contact.
- Insufficient understanding of English to follow the test instructions.
- Any suspected organic cause of psychosis (i.e., head injury, epilepsy or dementia).
- Taking antipsychotic medication for longer than 4 months before the start of the study.
Each participant was offered £10 for participating and if necessary, reimbursement for reasonable travel expenses (after producing a receipt). SU were recruited between 19/07/2018 – 23/05/2019. The study was reviewed by Research Ethics Committee (REC) and received approval from Health Research Authority (HRA) and Health and Care Research Wales (HCRW); project ID 236262, protocol number 1718/26, REC reference 18/SW/0065.
Control cohort (CC) was identified at UoE. In total we recruited 86 eligible participants, of which 43 played the same version of MG as SU and were used to identify the n=32 CC matching by age and gender the SU as close as possible. Participants were volunteers recruited by personally approaching potential participants, putting posters around UoE campus and at Exeter’s community centres, social media adverts, and snowball sampling. The exclusion criteria were:
- Moderate, or more severe, symptoms of depression, assessed by means of Patient Health Questionnaire-9 (PHQ-9).34 For ethical reasons we excluded the question concerning thoughts of suicide and self-harm. All participants scoring above 9, indicating at least moderate levels of depression, were signposted to several sources of support.
- A diagnosis of depression, an anxiety disorder or schizophrenia.
- Taking any psychopharmacological medication. Participants who indicated that they were having difficulties with mental health were directed to the UoE wellbeing centre.
- Suffering from seizures.
- English not being one of their first languages. This criterion was introduced to try to minimise the chances of misinterpretations due to the extensive use of questionnaires in the study.
CC was additionally screened using Community Assessment of Psychic Experiences-42 (CAPE-42);35 question 14, which asks about suicidal ideation and loads onto the depressive subscale, was excluded for ethical reasons. Each participant was offered £5 or one course credit for participating. CC was recruited by AW between 25/05/2018 – 26/11/2018 as part of his Master’s degree project. The recruitment of the CC was approved by University of Exeter, College of Life and Environmental Sciences (CLES), Psychology Ethics Committee, eCLESPsy000568 v2.1.
Participant flow chart is presented in Fig. 1. All participants gave written informed consent prior to the study.
Mirror Game
The mirror game (MG) used in this study was based on the algorithm described by Zhai and colleagues36 and followed closely our earlier work on establishing socio-motor markers of schizophrenia.18 MG is a movement task that can be used to assess socio-motor functioning (motor coordination and interpersonal synchronisation).18,19 We used two MG tasks. The first task was a Solo game, where participants were asked to move their hand freely in a horizontal direction. Participants were given the following instruction: “Please move your hand left and right, create an interesting motion and enjoy playing.” The second task was a Leader-Follower game. In the second task an animated image of a robot appeared on the screen. The animation showed the robot controlling its own dot. The dot moved horizontally according to a pre-generated movement pattern. Participants were tasked with following the dot’s movement as closely as possible whilst it was on screen. Participants were given the following instruction: “Please try to follow the movement of an animated robot as accurately as you can.” During the Leader-Follower game the robot was also presenting a parametric positive social feedback (smiling) as described by Cohen and colleagues.37
The two tasks were grouped into one session. The session consisted of the Solo game, three repetitions of the Leader-Follower game and another Solo game. Each game lasted for one minute. The session was repeated three times. Participants were free to take breaks between the games and sessions. Each Leader-Follower game used a different pre-generated movement pattern. Patterns were the same for each participant. We excluded the 1st Solo game and the 1st Leader-Follower game to allow participants to get familiar with the task. The SU participants were sitting in front of a 17”diagonal laptop computer (1100×680 pixels screen resolution), the CC was using a 23” diagonal computer monitor; the image displayed on the computer monitor was scaled down to use the central 17” diagonal part of the monitor and have the same resolution as the laptop display. Movement of the hand was recorded using a leap motion sensor38 and displayed as a dot on the screen. Participants used their dominant hand to control the horizontal position of a dot on the screen. The computer set-ups were different in the two groups due to the need of collecting the data simultaneously in multiple locations and additional research goals for the experiments with the CC that are not a part of the presented analysis.
In our analysis we used the recorded position of the participant hand (Solo and Leader-Follower) and the trajectory of the movement generated by the computer (Leader-Follower). Recorded position data is in arbitrary units in the range [−0.5, 0.5]; variable sampling rate, 90–140 Hz (S) and 40–70Hz (Leader-Follower). Pre-processing included:
- resampling to 100Hz with linear interpolation,
- low pass filtering with 5 Hz cut-off done using phase preserving Butterworth filter of degree 2,
- omitting the first and last 5s of the recording,
- estimation of movement velocity, using a fourth-order finite difference scheme.
Trail-making task
The trail-making task (TMT)16,39 is a valid, public domain test of visual attention and working memory, as well as executive control.40 It has two parts, which were alternated. In each part, participants must click on 25 dots in a specified order as quickly and accurately as possible. The visual attention part (TMT A) had participants click numbers in ascending order, 1–25. The executive control part (TMT B) had participants alternate between clicking on numbers and letters, both in ascending order (1-A-2-B-3-C etc.). Participants completed each part three times, alternating between TMT A and B, starting with TMT A. Only the last two repetitions were included in the analysis. We excluded the 1st repetition to allow participants to get familiar with the task. We used a digital version of the task implemented in PEBL: The Psychology Experiment Building Language.41 In the original study protocol the TMT was used as a non-diagnostic attention measuring task. It was retrospectively included in the analysis after literature review37,42–44 and data analysis indicated that including participants’ performance in this task could be beneficial for differentiating between the SU and CC.
For analysis we used the times between each individual mouse click made by the participant, we also used times between mouse clicks made on the correct targets.
Testing procedure
The stages of the research session are presented in Table 2. Both tasks and examples of collected data are shown in Fig. 2.
Sample size
For the feasibility study we recruited as many eligible SU as possible for the duration of the study. We approached all eligible participants that had been identified as appropriate for assessment for risk of psychosis by the Specialist Teams for Early Psychosis. Convenience sampling allowed us to proceed with the study as quickly as possible and assess what are feasible sample sizes for future research. Sample size of the CC was driven by the research objectives of AW’s Master’s degree project.
Features extracted from data
The selection of features for classification was informed by our earlier work,18 and modified to better fit the machine learning methodology employed in the current study. Instead of using distributions (histograms) as in the previous work, here we use a set of their properties (e.g., mean, standard deviation or median). As previously, the data was concatenated or averaged across the repetitions. Before averaging or concatenating the movement data, we remove parts where participants’ moves reached the edges of the sensor range (−0.5 or 0.5 value). The complete list of the features, and their description, is presented in Table 3.
Role of founding source
The founders had no involvement in study design; in the collection, analysis, and interpretation of data; in the writing of the report; and in the decision to submit the paper for publication.
DATA AVAILABILITY
The fully anonymised research data supporting this publication are openly available from: https://osf.io/rnzys/ [on acceptance the link will be changed into a permanent DOI identifier].
CODE AVAILABILITY
The code used for analysis presented in this publication is openly available from: https://osf.io/rnzys/ [on acceptance the link will be changed into a permanent DOI identifier].
RESULTS
Classes
We used the group (CC or SU), as the primary classification outcome (predicted variable), binary classification of the full dataset. Additionally, we used CAARMS score (CAARMS > 0 – at-risk mental state, psychotic or CAARMS = 0 – neither) as classification outcome of an independent binary classification within the SU group. CAARMS was completed by trained personnel of STEPs before completion of the MG and TMT tasks.
Classification methods
For classification we employed k-nearest neighbours (kNN) classifier.48 Specifically, we used a cosine distance to measure distances between the points in the 18-dimensional space defined by the 18 z-scored summary statistics measures (Table 3) and find the k-nearest neighbours. Cosine distance is defined as 1– cos(θ), where θ is the angle between vectors defined by the summary statistics measures coordinates and the origin of the coordinate system.
Additionally, we replicated the classification results using a Naïve Bayes (NB) classifier48 as implemented in Matlab 2022b function fitcnb49 with default settings. To avoid overfitting, we are using only three out of the 18 features (Table 3), namely one from the 6 measures estimated from the TMT data, one from the 4 measures estimated from the MG Solo task and one from the 8 measures estimated from MG Leader-Follower task. To select the features, we used the value of Cliff’s delta,50 a non-parametric measure of effect size. The set of classification features is chosen based on the results from the training phase, meaning that it is selected separately for each training-testing split (fold).
Training and testing
To evaluate the performance of the classifiers, we used two training-testing splits. A 25% hold-out (HO) training-testing split and a leave-one-out (L1O) training-testing split (corresponding to 2% hold-out in our case). Parameters and hyperparameters of the classifier are identified using only the training set. Hold-out data is used only for testing and is unseen by the classifier during the training.
In the 25% HO split we select at random 25% of the data (8 out of 32 participants in each cohort). We train the classifier using the remaining 75% of the data (24 CC and 24 SU datasets). We use the 16 participants (8 CC and 8 SU datasets) unseen by the classifier during training to construct confusion matrix, and compute performance metrics. To estimate 95% confidence intervals of the classifier performance we repeat the 25% HO split 1000 times (1000 folds).
The leave-one-out (L1O) training-testing split uses n-1 participants to train the classifier and 1 participant to test the model. L1O training-testing split allows to test the methodology n times. The L1O split simulates situation where we classify a new participant using classifier based on all the data available prior to the arrival of the new participant. To construct confusion matrix and compute performance metrics we compare the original classes with the set of individual predictions of each of the L1O splits, i.e., we compare original class of the participant unseen by the classifier with the class predicted by the model trained using the other n-1 participants.
Classification results
The proposed methodology allows classification of the CC and SU participants with an excellent (0.8-0.9)51 accuracy, sensitivity, specificity and precision. The only exception being the acceptable (0.7-0.8) sensitivity in case of the HO and kNN classifier. However, the methods fail to differentiate between SU with and without at-risk-mental-state (CAARMS score of 0 and CAARMS score > 0); binary classification within the SU cohort using CAARMS score as group label, CAARMS = 0 vs CAARMS > 0. Since there are only 16 participants with CAARMS = 0 and 16 participants with CAARMS > 0 we only use the L1O training-testing split. See Table 4 for details. To better understand the difference in performance of the proposed methodology in the two cases (CC vs. SU and SU CAARMS = 0 vs SU CAARMS > 0), we compared distributions of the summary statistics measures in the 3 groups. We did not find any statistically significant differences between the SU CAARMS = 0 vs SU CAARMS > 0 groups. Interestingly, we observed that the values of most summary statistics measures in the SU CAARMS > 0 cohort differ more from the CC compared to the difference between SU CAARMS = 0 cohort and CC; see Figure 3 and Table 5. Moreover, we observed that the effect size (Cliff’s delta) is overall higher for the summary statistics measures from the MG task while remaining relatively unchanged for TMT. The statistical significance of the difference observed in the pattern of change between the SU CAARMS = 0 vs SU CAARMS > 0 groups was investigated using a bootstrap test with 10000 random splits of the SU cohort of observing simultaneous low change in effect sizes for TMT (smaller than the median change of the 6 TMT summary statistics measures), mixed change in effect sizes for Solo MG task (larger than the median change of the 4 Solo MG summary statistics measures) and large change in effect sizes for Leader-Follower MG task (larger than the median change of the 8 Leader-Follower MG summary statistics measures). We found that this to be statistically significant with p < 0.0031.
DISCUSSION
We presented results of a feasibility study in which we investigated the potential for employing digital behavioural tests in healthcare practice for stratification of specialist teams for early psychosis (STEP) users and healthy cohorts. Our analysis demonstrated that the two investigated behavioural tests (MG and TMT) can be used to differentiate between STEPs users and healthy cohorts with excellent accuracy AUC=0.89 using two different classifiers k-nearest neighbours and naïve Bayes and two different training-testing splits, 25% hold-out and leave-one-out. Excellent performance of the classifiers is driven by statistically significant and large differences (large effect sizes) in summary statistics measures between the cohorts. Finally, we showed that cheap off-the-shelf equipment (laptop computer and a leap motion sensor) can be used to record clinically relevant behavioural data and that digital behavioural tests hold the prospect to aid clinical practice.
We also identified areas that require further research and development. We observed that the behavioural data from the MG and TMT collected in the current study cannot be used to differentiate between service users without (CAARMS=0) and SU with at-risk-mental-state (CAARMS=2) or first episode psychosis (CAARMS=4). This result might partially reflect limited specificity of the CAARMS assessment, meaning that only 15–22% of individuals with at-risk-mental-state develop a full psychotic disorder within 12 months.51–53 Another possible limitation is the small number of participants available in our SU cohort. Nonetheless, the fact the SU can be so accurately differentiated (large effect sizes for difference between summary statistics measures in the two cohorts) from the CC confirms that the so-called ‘non-cases’ among STEPs referral have a range of characteristic behavioural marker and constitute an important clinical cohort that differs from control cohort. 54–57 Moreover, the Specificity = 1 achieved by our proposed methodology (kNN classifier) means that it most accurately identifies control participants. This is important as misclassification in terms of mental health state in young individuals could have equally serious consequences due to stigma associated with mental health diagnosis.58
Furthermore, we showed (analysis of the effect sizes) that SU with CAARMS>0 differ more from the CC than SU with CAARMS=0. This indicates presence of differences between these two cohorts which could be uncovered by means of including additional tasks and additional data modalities. For example, recordings of hand movements during the TMT or recordings of eye-movements during both tasks. Inclusion of eye-movement data could be particularly beneficial since it is demonstrated to have diagnostic potential.24,27 Additionally, using mechanistic (differential equations) models59 to combine eye-movements, reaction time and movement data could help to identify people’s cognitive strategies e.g., employed to complete neuropsychological tasks.60,61 Identification of the cognitive strategies and understanding their causal mechanisms would elucidate the role of pathophysiology in perturbed information processing and allow the development of new methodologies for risk and treatment stratification.
Finally, longitudinal study using digital behavioural tests would be instrumental for understanding how motor coordination and other neurological signs change (decline or improvement) in the course of psychosis and why, as shown by Ferruccio and collegues,31 they allow to predict its long-term severity.
Study limitations
There are two main limitations of the study. First, we did not control the level of education in the two groups and it is know that the performance in TMT is affected by years of education.17 However, even using only the summary statistics measures from the MG allow to classify the CC and SU with AUC=90 (0.69-1) and sensitivity=0.75 (95%CI 0.5–1) and specificity=0.88 (95%CI 0.62-1); kNN, 25% hold-out and 1000 folds. The second, potential source of bias is the short exposure to antipsychotic medication; less than 4 months. We allowed 4 months of antipsychotic medication in order to facilitate recruitment of participants while minimising potential for manifestation of motor side-effects associated with anti-psychotic drugs.62 Therefore we anticipate the effect of medication status to be minimal and additionally confound with CAARMS score. In an earlier study we have shown that the obtained classification results are independent from anti-psychotic medication status.18 Furthermore, a recent study showed that neurological signs (e.g., tests of coordination and balance) and their change over 10 years is likely unrelated to exposure to anti-psychotic drugs.31
Implications for clinical and research practice
Our findings reinforce the benefits of digital behavioural test and quantitative analysis of their results and their potential for being used as a mobile assessment platform; assessable in home settings as well.63 Cheap, portable off-the-shelf equipment allows the assessment to take place in a range of indoor locations, while automatic data collection greatly simplifies the necessity for training.
Digital behavioural tests would benefit research on prognostic instruments for psychosis. Recent review51 identified heterogeneity in recruitment strategies for high-risk services as one of the factors limiting development of prognostic instruments for psychosis. Digital behavioural test could alleviate this limitation by stratifying and enabling the identification of more homogeneous clinical high-risk populations.
Finally, with further development standardised digital test batteries could supplement and augment neuropsychiatric/neurological tests making them quicker and easier to apply in routine clinical practice. Therefore, future work should focus on identification of optimal set of tests for establishing standardised digital batteries of behavioural tests as well as on longitudinal studies to understand how socio-motor function changes in the course of psychosis. This has the potential to be extended beyond STEPs users’ stratification64 and would facilitate rapid assessments for all people referred to mental health early intervention services,65,66 improving their quality of life and wellbeing.
Data Availability
All data and code used in the present work are available online at: https://osf.io/rnzys/
OTHER INFORMATION
Data Availability
The fully anonymised research data supporting this publication are openly available from: https://osf.io/rnzys/ [on acceptance the link will be changed into a permanent DOI identifier].
Code Availability
Code used for analysis presented in this publication is openly available from: https://osf.io/rnzys/ [on acceptance the link will be changed into a permanent DOI identifier].
Ethics approval and full study protocol
IRAS (Integrated Research Application System) ID: 23626.
REC (Research Ethics Committee) reference: 18/SW/0065, protocol number 1718/26.
University of Exeter, College of Life and Environmental Sciences (CLES), Psychology Ethics Committee: eCLESPsy000568 v2.1.
Further information about the study can be found at https://www.hra.nhs.uk/planning-and-improving-research/application-summaries/research-summaries/movement-and-perspective-taking-as-a-diagnostic-aid-for-psychosis/.
Full study protocol can be accessed at http://hdl.handle.net/10871/132205.
Contributions
Manuscript writing: P.S., K.T.A. Study design and methods development: P.S., S.S., T.E., Ph.S., A.K., K.T.A. Data acquisition and coding: P.S., A.W., S.L., J.W. Data analysis: P.S., K.T.A. Discussion of results and manuscript editing: All authors. Project coordination: P.S., K.T.A.
Competing interests
The authors declare no competing interests.
Sources of funding
The study was funded by: EPSRC Impact Acceleration Account, Impact & Knowledge Exchange Award, Jean Golding Institute seed corn award, Avon & Wiltshire Mental Health Partnership NHS Trust Research Capability Funding.
PS was generously supported by the Wellcome Trust Institutional Strategic Support Award 204909/Z/16/Z. KTA gratefully acknowledges the financial support of the EPSRC via grant EP/T017856/1.
The founders were not involved in study design; in the collection, analysis, and interpretation of data; in the writing of the report; and in the decision to submit the paper for publication.
For the purpose of open access, the authors have applied a ‘Creative Commons Attribution (CC BY) licence to any Author Accepted Manuscript version arising.
Acknowledgments
The research team would like to thank all the STEPs users and control participants who generously shared their time and took part in the project. This study would not be possible without them.