Prospective Registration of Trials: Where we are, why, and how we could get better
Division of Rheumatology, Geneva University Hospitals and University of Geneva, Geneva, Switzerland
Division of General Internal Medicine, Department of Medicine, University Hospitals of Geneva, Geneva, Switzerland
Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada
MAGIC Evidence Ecosystem Foundation [www.magicevidence.org], Lovisenberggata 17C, 0456 Oslo, Norway
*Corresponding author: Michele Iudici, Division of Rheumatology, Geneva University Hospitals and University of Geneva. Av. de Beau-Séjour 26, 1206 Genève. email: michele.iudici@hug.chABSTRACT
Objectives
Transparent trial conduct requires prospective registration of a randomized controlled trial before the enrolment of the first participant. Registration aims to minimize potential biases through unjustified or hidden modification of trial design. We aimed to (1) estimate the proportion of randomized controlled trials that are prospectively registered and determine the time trends and the factors associated with prospective registration; (2) evaluate the reasons for non-adherence with prospective registration and explore potential mechanisms to enhance adherence with prospective registration. We studied trials published in rheumatology as a case study.
Design and setting
We searched for reports of trials in rheumatology published between January 2009 and December 2022 using MEDLINE-PubMed. We retrieved trial registration numbers using metadata and reviewed full texts. We conducted a multivariable logistic regression to identify factors associated with prospective trial registration. We sent an online survey to authors of trials that were not prospectively registered. We inquired about possible reasons for non-adherence with prospective registration and asked about potential solutions.
Results
We identified 1093 primary reports of randomized controlled trials; 453 (41.4%) were not prospectively registered. Of these, 130 (11.9%) were not registered, and 323 (29.5%) were retrospectively registered. Prospective registration increased over time at a rate of 3% per year (p<0.001), with only 13.3% (2/15) trials prospectively registered in 2009 to 73.2% (112/153) trials in 2022. Even among journals publicly supporting ICMJE recommendation, 16% of the trials published in 2022 were not prospectively registered. In the multivariable model, prospective registration was associated with a larger sample size, recruitment conducted across countries, and publication in a journal with a higher impact factor. Trial evaluating non pharmaceutical intervention, especially education, delivery of health care or wellness, had a lower rate of prospective registration. Investigators reported lack of knowledge, or organizational problems as the main reasons for retrospective registration. Authors also suggested linking ethical approval to trial registration as the best option to ensure prospective registration.
Conclusions
Despite significant improvement, adherence to prospective registration remains unsatisfactory in rheumatology. Different strategies targeting journal editors, healthcare professionals, and researchers may improve trial registration.
Key findings
- Among 1093 randomized trials published in rheumatology between 2009 and 2022 30% were retrospectively registered and 12% not registered.
- Although the proportion of trials prospectively registered has increased over time, adherence remains suboptimal, even among trials published in journal endorsing ICMJE recommendations.
- Among the intervention tested by the trials, those concerned with education, delivery of health care or wellness had lower odds to be prospectively registered when compared to others.
What this adds to what is known related to methods research within the field of clinical epidemiology
- Although the amount of prospective registration has improved, our results raise the question about possible publication bias and other deleterious practices in research.
What is the implication, what should change now
- Solutions require attention at different levels, especially from researchers and journal editors. Prospective registration should be linked to the obtention of ethical approval, and reason for publishing trials not prospectively registered should be explicitly provided by the editors.
Article notes
Competing Interest Statement
TA is the Chair of the Magic Evidence Ecosystem Foundation. MI received consulting fees from Boehringer Ingelheim. Received payment or honoraria for lectures, presentations, speakers bureaus, manuscript writing or educational events from Boehringer Ingelheim and CSL Vifor, and has participated in the Advisory Board for Novartis. DM, DBG, SC, DSC and CG declare no conflict of interests.
Funding Statement
Swiss National Science Foundation. Grant number 212393.
Summary of Updates:
INTRODUCTION
Randomized controlled trials (RCTs) are key drivers of clinical decision making and progress in medicine1. Ensuring transparent conduct and dissemination of RCTs findings provide reliable knowledge, maximize patient’s health and, avoid the publication of misleading results 1,2. One of the pillars of transparent research is the process by which the protocol of a RCT is made publicly available prospectively, before enrolling the first participant 3. Its aim includes the promotion of pre-planned analyses following a defined hypothesis, the minimization of unjustified and hidden modification of trial design during the study course 4, and the reduction of deleterious practices in science such as p-picking, HARKING (Hypothesizing After the Results are Known), switching outcomes or selective outcome reporting 5,6. Trial registration, particularly when informed by a systematic review also helps map research in a field, understand intervention trajectories, grasp research context, and prevent unnecessary duplication of research 7.
The Declaration of Helsinki 8, and the International Committee of Medical Journals Editors (ICMJE) support prospective registration 3. The World Health Organization (WHO) considers registering interventional studies a ‘scientific, ethical, and moral responsibility’ 9. To be published in journals that are members or support the ICMJE 3, RCTs that started recruiting patients after September 2005 are supposed to provide a prospective registration on a free web repository. National or international clinical trial registries have been increasingly established throughout the world, with WHO centralizing their information on a Clinical Trials Search Portal 10.
Despite these efforts, failure to adhere to prospective registration is still an issue in medicine 11,12. As an example, a study found that less than 15% of RCTs published in the five top psychiatry journals have been prospectively registered 13. Even in the top ranking journal of medicine, 28% of the published RCT have been found to be not prospectively registered 11. In rheumatology, a recent study 16 evidenced that only 66% of rheumatology journals required a registration to consider a RCT for publication, but the prevalence of adherence to prospective registration in the field is unknown.
Several studies have proposed various factors associated with lack of prospective registration 14,15, among which the journal impact factor, if the intervention is pharmaceutical or not, the funding scheme, or the number of country involved in the study. But to date, details on the time evolution of prospective registration, and on how this practice differs between types of interventions or on the reasons for the lack of adherence are still missing.
In this study, we propose to study Rheumatology as a case study to assess the trends of prospective registration, the factors associated with the nonadherence with prospective registration, and explored the reason for such non-adherence and possible solutions to overcome it.
METHODS
Eligibility criteria of included studies
We included primary reports of RCTs in rheumatology published between 2009 and 2022 that started enrolling participants after 2005. We defined a RCT as a clinical study randomly allocating participants to different interventions. We excluded secondary publications of RCTs (post-hoc analyses, long term extension, secondary or additional analysis of RCTs), non-randomized or quasi-randomized studies, editorials, letters, protocols, erratum, corrigendum, systematic reviews, and studies not conducted on humans.
Search strategy of rheumatology RCT manuscripts
On February 1, 2023, using MEDLINE-PubMed and the strategy conducted by Al-Durra et al. 4, we searched all primary reports of RCTs published in the journals listed in the category “rheumatology” of the 2022 Journal Citation Reports of Clarivate 17, as well as all primary reports of rheumatology RCTs in the five top journals in general medicine (The Lancet, New England Journal of Medicine, JAMA, BMJ and Annals of Internal Medicine). Detailed search strategy and the list of included journals can be found in supplementary tables 1-2 and appendix1 - 2.
Selection of eligible manuscripts
Two authors reviewed titles and abstracts (SC and MI). If needed full texts were evaluated. Disagreements were resolved by a third reviewer (DM or DBG).
Data extraction
Identification of trial registration number
Presence of a Trial Registration Number (TRN) in each included study was determined automatically by first checking the PubMed-Medline metadata.18 After, the full text was reviewed using a list a regular expression patterns (Supplementary Appendix 3). If more than one TRN was found, one author (SC) checked each TRN against the registration data reported in the WHO-ICTRP database to identify the TRN corresponding to the publication. If no TRN was found in metadata or full text, the full text and supplementary material was reviewed. Also, an additional search on Google, Google Scholar and the Cochrane library was conducted. If no TRN was identified, the corresponding authors were contacted to inquiry about the TRN. If no TRN was found after these steps, the trial was considered as not registered.
Study outcome and definition of registration status
The primary outcome of the present study was the prospective registration of the RCT. A published RCT was considered prospectively registered if an associated TRN was found and had a registration date less than 30 days after the date of first recruitment.
Data Analysis
We analyzed the findings in two phases. First, we estimated the temporal trends of prospective registration of rheumatology trials and the factors associated with the failure of prospective RCT registration. Second, we investigated the reasons for non-prospective registration through an online survey sent to the contact authors of the trials that were not registered prospectively.
In the descriptive analyses, continuous variables were compared with t-test or Mann-Whitney test, and categorical variables with chi-square test or Fisher’s exact test, as appropriate. All statistical analyses were performed using R 4.2.0 and the xml2 library to handle xml files, the easyPubMed library to perform MEDLINE queries in PubMed and the rvest library to automatically recover data from websites.
Part 1: Prospective versus retrospective trial registration trends in rheumatology
Based on previous studies 11,19,20, we considered the following variables as potential factors related to registration practices: type of intervention, funding, sample size, countries of enrolment, year of the trial publication, journal impact factor, number of authors, and journal ICMJE status (for details on variables retrieval and operationalization, (Supplementary Appendix 4-5). The association between prospective trial registration and the chosen covariates was estimated using univariable and multivariable logistic regression. Linearity of the association between prospective trial registration and sample size and impact factor was assessed visually. We considered for both covariates a piecewise linear relation with prospective registration composed of two segments, and the knots were estimated using segmented modelling 21. Missing covariates were handled with multiple imputation with chained equation, using 20 samples and 5 iteration and using the outcome and the ensemble of covariates in the model. Estimates of regression were pooled according to Rubin’s law 22..
Part 2: Reasons for noncompliance with prospective registration in rheumatology
An online survey was sent to the corresponding authors and/or to the designated trial contact persons of trials not registered prospectively. The questions used in our survey (Supplementary Appendix 6) were based on the survey published by Hunter et al. (17) and aimed at investigating the reasons for nonadherence with the prospective registration and explore the investigators’ opinion about possible measures that could have prevented this issue. The online survey was sent and stored using the electronic data capture system (REDCap, Vanderbilt University, Nashville, TN, USA) 23.
Ethics
The Geneva Research Ethics Committee Ethics committee exempted the present study from formal ethics review since we used publicly available data.
RESULTS
Identification of Trial Registration Number (TRN)
We screened 1,707 records and identified 1093 primary reports of RCTs, 41.4% (453/1023) of them were not registered prospectively. Among these 11.9% (130/1093), had been not registered and, 29.5% (323/1093) were retrospectively registered) (Supplementary Table 3).
Characteristics of the included trials
Most studies were conducted in a single country 79.7%, (872/1093) and were not industry-funded 57.8% (632/1093), The majority of included RCTs assessed pharmacological interventions 42.6% (466/1093 42.6) followed by interventions related to exercise therapy 13.9% (152/1093) and procedures 13.6% (149/1093). About half of the included studies were published in journals endorsing the ICMJE recommendations 51.4% (562/1093). From the included trials, 40.0% (421/1093), were funded by the industry and focused on pharmacological interventions 73.6% (310/466) (Table 1). Among RCTs published in journals endorsing ICMJE recommendations, 27.5% (155/562), were not prospectively registered, the number decreased in time to reach 16% (12/75) in 2022. In the overall sample, 21.7% (122/562) of the RCTs were retrospectively registered, this number also that decreased over time. Around5.9% (33/562) of them were not registered at all, a proportion that remained stable in time (Supplementary Table 4). Part 1: Prospective versus retrospective trial registration trends in rheumatology
When compared to non-prospectively registered RCTs, prospectively registered RCTs were more often conducted in more than one country 92.5% (406/453) vs 74.7% (466/640), assessed more frequently pharmacological interventions 51.6% (330/640) vs 30.0% (136/453), among those registered prospectively 48.4% (310/640) were industry funded. Also, prospectively registered RCTs had a larger median sample size (148 vs 80 participants), had more authors (median 7 vs 10 authors), were more frequently published in ICMJE journals 63.6% (407/453) vs 34.2% (155/640) and, had a higher median impact factor (4.52 vs 2.48) (Table 1). The proportion of prospectively registered RCTs increased over time at a rate of three percentage points per year (p<0.001), with less than one-third of trials prospectively registered in 2009, against 73.2% in 2022 (Figure 2 and Supplementary Table 4).
All these factors were significantly associated with prospective registration in the univariable analysis (Table 2). The association of prospective registration and either impact factor or sample size (Supplementary Figure1) showed a clear two pieces linear relationship, with a knot at 500 [95%CI: 108-892] for sample size and 5.5 [95%CI: 3.3-7.6] for impact factor. The sample size increased the odds of being prospectively registered when below 500 participants (OR 1.46 95%CI [1.32, 1.60] for every 100 participants) but had no effect on prospective registration when above 500 participants (OR 0.99 95%CI [0.98-1.02]). Similarly, the impact factor increased the odds of prospective registration when the impact factor was below 5.5, (OR 1.54 95%CI [1.39, 1.71]) but this association was weaker when the impact factor was above 5.5 (OR 1.10 95%CI [1.08, 1.12]).
In the adjusted model, the factors that remained independently and positively associated with prospective registration were recruitment was done across multiple countries(OR 1.84 95%CI [1.12, 3.01]), the impact factor when it was below 5.5 (OR 1.38 95%CI [1.18, 1.61]), the planned sample size when below 500 participants (OR 1.21 95%CI [1.09, 1.35]), and the year of publication, (OR of 1.16 95%CI [1.11, 1.20] per year).
Regarding interventions categories (eTable5), educational interventions (OR 0.64 95% CI [0.14, 0.79]), interventions associated to the delivery of health care (OR 0.57 95% CI [0.34, 0.97]), and wellness and spa interventions (OR 0.24 95% CI [0.08, 0.70]) significantly reduced the odds of prospective registration. Regarding health conditions, RCTs assessing interventions for knee pain, neck pain or low back pain (OR 14.14 [2.74, 73.06], 8.01 [2.06, 31.05], 3.56 [1.69, 7.50]), interventions for fractures (OR 3.51 [1.27, 9.70]), interventions related to arthroplasty (OR 3.11 [1.51, 6.39]) and to osteoarthritis (OR 1.99 [1.26, 3.15]) had a significantly higher probability to be prospectively registered when compared to RCTs in rheumatoid arthritis (Supplementary Table 6).
Part 2: Reasons for non-adherence with prospective registration in rheumatology
Retrospective registered trials
We sent 365 survey invitations (296 corresponding authors, 69 clinical trial contacts) and, 42 accepted to participate (response rate 11.5%). Compared to non-respondents, respondents were involved in studies more frequently published in journals endorsing ICMJE recommendations (47.6% vs. 32.9%), and with a higher impact factor (Supplementary Table 7). The main reasons for retrospective registration were a lack of knowledge (45.0%), either about the need for prospective registration in general or, less frequently, about the type of trial requiring prospective registration (Figure 3, left panel), and time constraints (17.0%). As far as potential factors that could prevent this issue, respondents suggested linking the obtention for Ethics approval to trial registration as the best option to ensure prospective registration (Figure 3, right panel).
Unregistered trials
We sent 126 survey invitations to contact authors and only 14 authors accepted to participate (response rate 11.1%). Although most of the characteristics of the trials were similar among respondents and non-respondents, a higher proportion of non-respondents published their RCT results in journals endorsing ICMJE recommendations (75.9% vs. 50%) (eTable8). The main reason for non-registration was lack of awareness (6 out of 7 responses). Similarly, respondents suggested having the registration as part of the requirements for Ethics approval or a mandatory requirement from scientific journals to ensure prospective trial registration.
DISCUSSION
Our study included over 1,000 RCTs published in rheumatology journals from 2009 to 2022. Of these 41.4% were not registered prospectively, with 29.5% retrospectively registered, and 11.9 % not registered at all. Over time, registration practices gradually improved. The factors associated with better trial registration were recruitment conducted across countries, a higher sample size among small trials (below 500 participants), publication in a journal with a higher impact factor and trials investigating pharmacological interventions. The lower odds of prospective registration among non-pharmacological interventions were mainly caused by intervention concerned with education, delivery of health care and wellness.
Despite efforts supporting trial registration, and although registration of trials is mandatory since 2005 in the United States and Europe, adherence to prospective trial registration remains inadequate in rheumatology. The gradual improvement in registration is slow, making adherence an unsolved issue in medicine. Adherence to prospective trial registration in the Australian New Zealand Clinical Trials Registry (ANZCTR) was 48% in 2006 and 64% in 201519. Lower rates of prospective trial registration have been reported in a study analyzing a large sample of clinical trials included in 100 Cochrane reviews (only 31% were registered, of which just one-third prospectively) 15 in surgical trials 50% 24, and in trials investigating drugs for the prevention of postoperative nausea and vomiting 36%25. Also, a large study analyzing the registration status of more than 10’000 trials found adherence with prospective registration in 42% of cases 4. Comparing our findings with those from other medical specialties is challenging due to variations in study eligibility criteria, investigation periods, and the definition of prospective registration 26. We considered a trial as prospectively registered if an associated TRN was found with a registration date less than 30 days after the date of first recruitment, to incorporate also the FDA regulation that requires all submitted trials to be registered no later than 21 days after enrolment 27, and to take into account those trials whose registration/start date were provided at month level. However, when other studies applied a more conservative definition of prospective registration (as we did), the observed adherence to registration requirements were comparable to our results 28. Altogether, these findings underline that a high proportion of trials, whose results drive daily practice, may suffer from deleterious and difficult to identify and amend issues such as publication bias, selective outcome reporting, or data-driven analysis. These practices are associated with the risk of yielding false-positive results, potentially leading to the implementation of ineffective or harmful measures29 and, finding effective solutions to this issue is a paramount concern for public health.
Though adherence to prospective registration among trials published in journals endorsing ICMJE guidelines was considerably higher, 16% of RCTs were still accepted without prospective registration, highlighting that some journals do not consistently adhere to their registration policies. The reasons for accepting and publishing unregistered or retrospectively registered trials might be related to hesitance to penalize valid research or studies from developing countries, concerns about losing submissions to competing journals, or misunderstandings about the relevance of registration policies 30. Although this is rarely disclosed, it would be relevant that journals accepting trials that either did not register or did it retrospectively state their reason for accepting them 25,31.
The reasons for non-prospective registration provided by the authors who answered our survey were consistent with other studies. Reveiz et al. 32, two years after the publication of ICMJE recommendations on trial registration, found that one-third of the researchers identified the lack of knowledge as the predominant factor for not registering trials prospectively. More recently, a survey conducted by Ranawaka et al. 33 among participants in a medical congress, including active researchers, further highlighted suboptimal awareness regarding the necessity and perceived importance of prospective trial registration. Developing a procedure enabling researchers to register RCTs prospectively as part of the submission to the Ethics Committee appears as a potential solution, already implemented in some countries 19. Besides the lack of awareness about registration, other factors such as the time needed to register, cost and complexity of registration, and generic ‘organizational’ problems have also been identified as significant barriers to registration 19,34. Strategy prioritizing user-friendliness, speed, and affordability of the registration process may help increase the rate of prospective registration. Further measures may include legal frameworks and active enforcement of a prospective registration by funding bodies, ethical committees, and institutions, and of course the implementation of teaching modules emphasizing the relevance of study registration during medical and healthcare curricula.
Limitations and strengths of this study
Some limitations should be acknowledged. First, we did not review the full text of RCTs to identify the reasons for retrospective or non-registration. However, we consider this a minor concern given that a recent study showed that only 3.5% of RCTs report the reasons for retrospective registration in the published manuscript 35. Second, there is a risk of selection bias due to the low proportion of participants in our survey that intended to determine the reasons for non-prospective registration, as respondents and non-respondents might have different profiles.
The strengths of our study include the broad time interval considered together with the amount of information used in the analysis, which makes it one of the most complete studies on the subject and the first one to address this relevant issue in Rheumatology. Also, the direct feedback from investigators about reasons for lack of registration or retrospective registration and their opinions on potential solutions provides better qualitative insights on adherence to registration.
Conclusion
Despite the observed improvement over time, adherence with legal and ethical obligations of prospectively registering trials is still suboptimal in rheumatology. Even journals endorsing the ICMJE recommendations on registration practices still publish unregistered or retrospectively registered studies. As a result, a high proportion of trials whose results drive daily practice are likely to suffer from deleterious and difficult to identify issues such as publication bias, selective outcome reporting, or data-driven analysis. All these practices may contribute to spurious inferences about current best care, potentially leading to the implementation of ineffective or harmful measures (24). Our results strongly support the need to for implement educational and regulatory strategies to prevent these issues.
Supporting information
Data Availability
Code and data are available at the following Gitlab repository:
https://gitlab.unige.ch/trial_integrity/prospective_registration_public
Conflicts of Interest and Financial Disclosures
TA is the Chair of the Magic Evidence Ecosystem Foundation.
MI received consulting fees from Boehringer Ingelheim. Received payment or honoraria for lectures, presentations, speakers bureaus, manuscript writing or educational events from Boehringer Ingelheim and CSL Vifor, and has participated in the Advisory Board for Novartis.
DM, DBG, SC, DSC and CG declare no conflict of interests.
Funding
Swiss National Science Foundation. Grant number 212393.
Role of the Funder/Sponsor
The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Data sharing statement
Code and data are available at the following Gitlab repository: https://gitlab.unige.ch/trial_integrity/prospective_registration_public