This study used a meta-analytic approach to examine the magnitude of PM impairments across studies of various licit and illicit recreational drugs. Our analyses revealed that regular users of alcohol, cannabis, ecstasy and opiates performed significantly worse than controls on event- and on time-based PM tasks. Regular smokers performed significantly worse than controls on event-based but not time-based PM tasks, although the effect size estimate for the latter was particularly imprecise with the true effect lying between very large (SMD approximately − 2) and negligible. The effect sizes for impairments on event-based PM tasks were moderate–large in smokers and heroin users (the latter based on a single study), small–large for alcohol, ecstasy and methamphetamine use, and negligible–large for cannabis. Effect sizes for time-based PM tasks were moderate–large for methamphetamine and opiate use, moderate for cannabis, small–large for ecstasy and negligible–large for tobacco and alcohol. Heterogeneity influenced a number of effect size estimates. As an illustration of the effects of individual studies in determining heterogeneity (as well as overall effect size), exclusion of the study by Platt et al. (2016), resulted in a marked reduction in the heterogeneity of effect sizes in time-based PM performance amongst studies of heavy drinkers, as well as nearly halving the overall effect size.

Despite reflecting the effects of chronic/sub-acute substance use, rather than acute intoxication, the effects summarised here are broadly in line with laboratory studies of acute effects of alcohol on PM (Leitz et al. 2009; Montgomery et al. 2011; Paraskevaides et al. 2010). However, it is important to note that effects of acute alcohol have to date only been investigated in light/social drinkers. The acute-on-chronic effects of alcohol in heavy drinkers have yet to be investigated. In addition, effects reported here on ecstasy (unverified MDMA content) are similar to those found following acute MDMA administration in laboratory settings (Kuypers et al. 2013; Ramaekers et al. 2009; van Wel et al. 2011). In contrast to the effects of smoked tobacco/nicotine reported here, laboratory-based studies on the acute effects of nicotine have found nicotine to improve PM performance of smokers (Dawkins et al. 2013; Jansari et al. 2013; Rusted et al. 2005; Rusted and Trawley 2006) and ‘non-smokers’ (Rusted and Trawley 2006), with the exception of one study showing no effect of nicotine on PM in never-smokers (Jansari et al. 2013).

A number of methodological issues were identified within the reviewed studies. For example, there was a high level of polydrug use across studies, particularly in the ecstasy studies. This highlights the difficulty of distinguishing discrete drug effects in studies of non-acutely intoxicated users. Specifically, it is unclear to what extent the observed impairments were due to the primary drug of interest versus other regularly used drugs, or indeed the combination of drugs. In studies of smokers for instance, participants had high levels of alcohol use, some reaching levels as high as those reported in the studies of heavy drinkers. The lack of comprehensive reporting on general drug use in many studies makes it difficult to ascertain the true effect of polydrug use on the effects reported here. In addition, since assessment of bias relating to drug use in the previous 24 h was ‘unclear’ in most studies, the role of abstention (or withdrawal) or acute drug effects in the impairments reported here also remains unclear. Related confounds also complicate interpretation. For example, Ludicello et al. (2011) reported high prevalence of AIDS in their sample of methamphetamine users (60%) introducing two potential confounds; the cognitive impairment of HIV/AIDS (Watkins and Treisman 2015) and the possible effects of antiretroviral therapy on cognitive performance (Liner et al. 2008).

Many studies had recruited participants from student populations. Higher levels of education and/or cognitive ability (intelligence) in the drug-using groups may have protected against the cognitive impairing effects of drug use (in line with a cognitive reserve hypothesis) and contribute to some of the smaller effect sizes found in the meta-analyses. Other studies recruited their drug-using groups from outside student subject pools and in clinical settings or detox treatment (e.g. Bedi and Redman 2008b; Griffiths et al. 2012; Terrett et al. 2014; Weinborn et al. 2011). For example, Griffiths and colleagues recruited their experimental group from a residential substance misuse service for those with alcohol dependence, and their control group from a university subject pool consisting of students and members of the public. These studies might have greater clinical relevance but they also have a potential for greater bias, associated with for example, the presence of comorbid psychiatric disorders (Hasin et al. 2007) and lower socioeconomic status (Hackman et al. 2010) in more severely affected drug users found in clinical settings. For example, schizophrenia is associated with greater PM impairments than affective disorders (Burton et al. 2018). By recruiting comparison groups from the same or similar populations, matching groups on relevant demographics (e.g. age, educational achievement, premorbid IQ) and using conservative inclusion criteria, future research should aim to increase the internal and external validity of studies.

Surprisingly, lifetime drug dose did not appear to moderate the size of PM impairments. This may have been a result of the relatively crude categorical rating scheme used here, as lifetime dose is a continuous variable. However, this was chosen given the variance in reporting methods on quantifiable drug use. An alternative categorical rating scheme might have been more informative, examining the moderating influence of regularity of drug use (e.g. daily versus less regular use). Moderation analyses within each drug group might have been more informative in determining the moderating role of lifetime drug dose, since cumulative neurotoxicity may be more severe for some drugs (e.g. alcohol and methamphetamine) than others (e.g. smoked tobacco). However, given the small number of studies per drug grouping, such an analysis would have been underpowered.

In most studies, participants were allocated to groups with un-validated measures of drug use. The exception was Weinborn et al. (2011) and Platt et al. (2016). In these studies, a cut-off score on a validated measure of alcohol use was used for group allocation. However, no further verification of drug/alcohol use was used and as such, healthy controls may have under-reported the quantity and frequency of drug/alcohol use (e.g. see Townshend and Duka 2002). Such under-reporting might have reduced the differences in mean PM performance between control and drug-using groups.

There are a number of limitations inherent in the present review. First, as already noted, the number of studies per drug class was relatively small, which may have adversely affected the precision of effect size estimates. Second, different measures of PM were used across the different studies, with some measures having limited evidence of adequate psychometric properties. Thirdly and relatedly, the effect sizes were characterised by moderate to high levels of heterogeneity, limiting the strength of conclusions about true effect sizes. Fourthly, we limited the inclusion of studies to those covering relatively common drugs associated with problematic use. Other major drug classes, including various novel psychoactive substances that are known to have cognitive impairing effects, were not examined.