The observational studies [8,9] identifying a potential drug interaction between cannabis and immunotherapy have been widely cited, including in clinical practice guidelines [10,11] and in a publication geared to the general public [2] (Table S1). This report determined that an appreciable subset of the statistics contained in [8,9] could not be verified, as was generally consistent with the PubPeer reports [13,14,15]. There are a few possibilities that might be able to account for only 4.5% of the p-values in the correction [9] being verified and the many inconsistencies. Analyses in the retrospective report were completed with SPSS, version 21 [8]. The prospective study listed both SAS and R (https://www.r-project.org/ accessed 18 March 2024) [9]. Although we used GraphPad for our analyses [25,27], we feel that it is unlikely that different software would produce such disparate results. Another possibility is that research with patients is challenging, and there was a small amount of missing data. We feel this explanation is also unlikely because the correction listed the N for each cell for four variables [9], and there were no missing data. Similarly, verification of the percentages (File S2) did not indicate that the denominator was decreased. However, we cannot discount this possibility for other variables or tables. Another possibility, which we feel is more likely because the p-values corresponded to four decimal places on fifteen occasions (Table 1), is that the methods section listed one statistic (chi-square), but a slightly different statistic (chi-square with Yates) was completed. The PubPeer response by a middle author also indicated that Yates was completed [14]. Notably, the BMJ guidance [20] attempted to clarify when to use chi-square, Fisher’s, and Yates. Table 1 suggests that Yates may have been uniformly used, independent of the minimum expected value or the total N. Although the methods section listed “We computed 2-tailed p-values”, as the attempted verification produced p-values that were approximately twice as large (and the results section alluded to a one-tailed test), it is also possible that the reported p-values were one-tailed in the previously published Table 2 [9]. Similarly, while the methods section [8] states that “Chi-square test was used to determine the difference between patients’ characteristics in both groups.”, it is likely that some of the reported p-values were from Fisher’s. It is important for the readers to be informed of which nonparametric statistic was completed so that they can assess the Type I error rate. There is a general consensus that Yates is overly cautious in its desire to avoid a type I error [19]. Fisher’s should not be reported if the total N of all four cells is above 40 (the N was 140 in [8]) and the expected N was ≥5 [21]. Although choosing which non-parametric statistic to use can be challenging when different resources have contradictory recommendations [19,20] (File S1), at the very least, we believe that telling the audience in the methods that you will run statistic A but then reporting in the results the findings from statistic B is a non-trivial oversight.

In evaluating the prior findings of a possible cannabis–immunotherapy interaction [8,9], it is important to be cognizant that there were higher rates of lung, laryngeal, esophageal, pancreatic, urinary, stomach, hepatic, colorectal, renal, bone marrow, and cervical cancers in patients who smoked [12]. Smoking tobacco introduces the human body to a complex interplay between thousands of chemical constituents, including dozens of known carcinogens [28,29]. The dangers of this chemical cocktail are well documented, as 40% of new cancer diagnoses are attributed to cigarette smoking [30]. Furthermore, smoking not only increases the relative risk of developing certain cancers, but it has also been associated with poor treatment outcomes [31]. A retrospective review of 439 patients with non-small cell lung cancer found heavy smoking to be associated with poorer overall survival (hazard ratio (HR) = 1.4). This relationship existed regardless of cancer staging [32]. Reported relationships between smoking and melanoma are inconsistent, with some reports demonstrating a paradoxical decreased risk of melanoma in smokers [33,34,35]. Despite this, a cohort study of 6279 patients with stage 1 or stage 2 melanoma reported a greater risk of melanoma-associated death in current smokers (HR = 1.5) [36]. A meta-analysis including 1326 cases of renal cell cancer (RCC) from cohort studies and 6032 cases of RCC from case–control studies reported a relative risk of 1.4 for the development of RCC in ever-smokers [37]. Furthermore, a systematic review and meta-analysis including 343,993 cases of RCC reported that current smoking was associated with increased disease-specific mortality (HR = 1.5) and worsened progression-free survival (HR = 2.9) [38].

Although there were many differences between our findings are those reported earlier [8,9], we are not suggesting that anyone has engaged in anything nefarious. We commend the earlier reports [8,9] for addressing a timely and important topic. However, it may also be valuable to prevent situations like this in the future. If the authors are going to engage in an atypical practice (e.g., floor rounding [14]), it would be beneficial to briefly document this in the methods section. It is also possible that floor rounding was employed for reporting the p-values. About one-third of leading biomedical journals have reported rarely or never using specialized statistical reviews [39]. As the statistical analysis section contained “A series of χ2 tests or Fisher’s exact tests (when the assumptions of the parametric χ2 test (sic) were not met)” [9], if there was a statistical reviewer, this person may not have been reading carefully. Emails to the editors asking whether these manuscripts [8,9] received a statistical review did not receive an affirmative response. Further, it would be well beyond standard practice to expect a reviewer (often a volunteer) to re-run all the analyses. In psychology, it is common to list the test statistic, the degrees of freedom, and the p-value. An analysis of a quarter million psychology papers revealed that half contained inconsistencies where the reported test statistic and p-value did not correspond. Further, one-seventh of papers included gross inconsistences defined as “the reported p-value was significant and the computed p-value was not, or vice versa” [40]. Some experimental psychology journals use software to identify statistical irregularities [41]. We are not aware of any biomedical journals which are employing similar software to assess percentage calculations, including rounding errors or non-parametric analyses.

Age was not reported as statistically significant in [8], but had a significant difference (p < 0.002) in our analysis (Figure 1A). This age difference could reflect that cannabis is psychoactive and providers in Israel may be less likely to prescribe cannabis to older patients [42]. Alternatively, cannabis users could have had a more aggressive or more advanced form of cancer at baseline. As the cannabis users were much more likely to have liver metastasis at baseline (Figure 1D), cannabis may simply be a proxy for a high-burden symptomatic disease [41]. Whether immunotherapy was received as a first- or second-line treatment was reported as non-significant (p > 0.050) in Table 1 [8], and again in [9]. However, re-analysis revealed that this baseline difference was significant (Figure 1C). Both of these findings could be interpreted as gross inaccuracies. In one high-profile instance of a manuscript with multiple issues, there was a twelve-year interval between the original MMR vaccine and autism study [43] and the retraction [44]. Journals, including those with reasonable open-access fees (e.g., Cancers is currently CHF 2900 [45]; The Oncologist is currently USD 3669 [46]), need to have sufficient staff to deal with these concerns in a timely fashion. A practice that might be helpful moving forward is to include the full data and a detailed data dictionary as supplementary material, either with the published manuscript or with a preprint, to allow a second party to verify all analyses. The author instructions of Cancers include “We encourage all authors of articles published in MDPI journals to share their research data.” [46]. Similarly, the author guidelines for the retrospective journal [47] notes that “The Oncologist strongly encourages authors to make all data and software code on which the conclusions of the paper rely available to readers.” An email to the corresponding author requesting the original data was not responded to. It would also be informative to list the N for all cells in each analysis in a Supplementary Text S1 to facilitate statistical verification.

Basic science investigations are crucial, as they can avoid the many potential confounds (e.g., age, disease severity, smoking [12], Figure 1) that are challenging to overcome with observational reports. A recent report attempted to replicate and extend the studies [8,9] in two ways. First, tetrahydrocannabinol did not impact the enhanced survivability of anti-programmed death ligand 1 antibody treatments in a murine colorectal model. Second, although the advanced non-small-cell lung cancer patients using cannabis (N = 102) were significantly younger, more likely to have brain metastasis, and marginally more likely to have liver metastasis (p = 0.06) at baseline than those that did not (N = 99), cannabis did not significantly impact the survivability following pembrolizumab [42].

In balancing the benefits and harms of cannabis for cancer patients [1,2,42], a broader perspective is useful [48,49,50,51,52,53,54,55]. There has been appreciable work focused on the anti-cancer activity of cannabinoids, including possible use in potentiating immunotherapy [48,49,50,51,52]. Cannabinoids, including cannabigerol (CBG), cannabidiol (CBD), and tetrahydrocannabinol, have been shown, in vitro and in vivo, to inhibit cancer cell proliferation and metastasis, while also promoting apoptosis and suppressing cancer-related angiogenesis [49,50,51]. Cannabinoids may also play a positive role in regulating aberrant cellular metabolism, which is characteristic of cancer [51]. With respect to the role of cannabinoids in potentially potentiating immunotherapy in colorectal cancer, these agents may boost immunogenicity through cytotoxic effects on cancer cells and the subsequent release of antigens, as well as reprograming immune cells to target tumors [51]. Given the various mechanisms by which cannabinoids may have anti-cancer effects, it is important not to dismiss potential benefits based on possibly unverifiable statistical interpretations of data. In addition, the Taha [8] and Bar-Sela [9] studies involved a limited number of cancer types, which should not be considered universally applicable for cancer immunotherapy in general. It should also be noted that effects of cannabinoids on immunotherapy may not be influenced only by cancer type, but also by the concentration of the relevant agents. For example, while higher concentrations of THC had immunosuppressive effects in vitro and in vivo, lower concentrations were immunostimulatory [53]. All of these facts recommend caution in the interpretation of the Taha [8] and Bar-Sela [9] studies, as well as other naturalistic investigations where the cannabis route of administration and THC concentration are not homogenous.

The ability of cannabinoids to influence carcinogenesis and response to immunotherapy, particularly with respect to the colon, could be modulated by nutritional factors, synthetic organic agents, and the gut microbiota. For example, bioactive plant compounds such as flavonoids affect colon tumorigenesis and anti-cancer therapeutics [56]. Xanthohumol demonstrated greater anti-tumor effects on colorectal cancer (CRC) cell lines than those of the chemotherapeutic agent 5-fluorouracil (5-FU); while apigenin and luteolin demonstrated lesser anti-tumor effects, these flavonoids exhibited positive synergism, boosting the efficacy of 5-FU [56]. Since these agents are also known to reduce side effects of CRC chemotherapy, they have a double utility, both promoting anti-tumor activity and making chemotherapy more bearable for the patient. More generally, bioactive food compounds, including those associated with the Mediterranean diet, can affect cancer-related microRNAs (miRNAs) to suppress tumorigenesis, while high-fat diets have an opposing, pro-tumorigenic effect [57]. Given the possible anti-tumor effects of cannabinoids and their use to reduce the side effects of cancer treatment, positive interactions between cannabinoids and nutritional supplementation need to be explored, including with respect to cancer immunotherapy. Other agents may be important adjuncts to CRC therapy in conjunction with cannabinoids. For example, upregulation of the ion channel protein TRPM8 is associated with a poor outcome in CRC patients, and the TRPM8 ligand WS12 represses colon tumorigenesis in mice, at least in part through downregulation of canonical Wnt signaling [58]. Thus, synthetic organics like WS12 may synergize with cannabinoids in the treatment of CRC.

The gut microbiota can influence the development of gastrointestinal cancer, and differences in the microbiota are associated with the up- or downregulation of a number of non-coding RNAs, including miRNAs, linked to the development of gastric or colon cancer [59]. As bioactive food compounds can also alter cancer-related miRNAs [57], and diet can alter the microbiota itself, it is likely that diet and the microbiota synergize to influence CRC risk. Further, interactions between the gut microbiota, dietary fiber, and colon tumorigenesis have been identified [60]. Thus, diet and nutrition synergize with the underlying microbiota to affect tumorigenesis, and these effects could influence cannabinoids’ effects on tumorigenesis and on treatment (e.g., immunotherapy), as well as on the adverse effects of anti-cancer treatments (e.g., nausea). Therefore, when considering the question of cannabinoids and cancer, the context of the patient’s microbiota, as well as nutrition, are variables that must be taken into careful consideration. An incomplete understanding of these factors, or a failure to account for these variables, can contribute to misleading conclusions with respect to the negative effects of cannabinoids on immunotherapy and other cancer treatments.

One limitation of this report is that only a subset of the originally reported findings could be assessed for veracity because the full raw data were unavailable. Therefore, we cannot infer whether the figures contained verifiable, misreported, or unverifiable information. As noted above, we cannot discount the possibility that the Israeli team [14] also used floor rounding to report their p-values (e.g., p = 0.0896 reported as 0.08), so some analyses that were classified as unverified may in fact have been misreported instead. There are many statisticians who would argue that it is ridiculous to interpret the results of a study with p = 0.055 differently than one with p = 0.045. Dividing the results of hypothesis tests into “significant” and “non-significant” is both unhelpful and outdated [52]. Although we would concur with the thrust of this argument, findings like Table 1 [8], where age is described as non-significant (i.e., p > 0.05) but then has a recalculated p-value of 0.0021, or the liver metastasis, which was reported as “p = 0.89” but calculated as p ≤ 0.0001 [9], are more concerning, as they obscure the reader’s ability to accurately determine whether the cannabis user and non-user groups were equivalent at baseline.