More and more general population surveys using representative samples are attempting to assess adult drug use in European countries. In France, various surveys have been conducted since the 1990s. The National Health Barometer (HB) is the primary source of information about prevalence, correlates, and trends in substance use and misuse in France [1]. This survey collects data every five years from a nationally representative sample on patterns and correlates of licit and illicit drug use and related problems, with a focus on cannabis use. It is a telephone-administered interview survey. The last but one iteration of this cross-sectional survey was conducted in 2005. A few months later, the Life Events and Health Survey (LEHS) was conducted, a face-to-face interview survey mainly dealing with violence and including an audio computer-assisted self-administered section (the A-CASI) to assess licit and illicit drug use [2]. The questions in the two surveys were the same, based on the European Model Questionnaire elaborated in the late 1990s [3]. Both surveys generated prevalence estimates for lifetime substance use and past-year substance use. The aim of this paper is to compare the results of these two national surveys in order to assess the impact of the technologies used on the measurement of sensitive issues.

Although a comparison between two French surveys among adolescents in the late 1990s has already been published [4], this is the first time that such a comparison for the whole population has been possible in France, with two large samples and two different data collection methods. The availability of these two large surveys carried out during the same year with nationally representative samples presents a unique opportunity to address the issue of data collection mode effect.

A number of studies in the literature have shown how methodology can influence prevalence estimates for alcohol [5]–[8] or drugs [9], [10] in the adult population, as well as among adolescents [11]–[13]. Although it seems to be difficult to draw conclusions about areas of disagreement in comparisons of this type, most authors agree that increased reporting of substance use is a sign of improved validity in the methodology (because these behaviours tend to be underreported). When the comparison is done on data collected in the same period of time, increased reporting should be attributed to the data collection mode implemented as it cannot be a real increase [11], [14].

A large number of studies over several decades have fairly clearly underlined that self-administered questionnaires are more suitable than other data collection modes for collecting reports on sensitive behaviours [5], [9], [14]–[21]. This result is particularly convincing among adolescents and young adults [22]–[24]. Indeed, answers given to self-administered questionnaires, whether they are completed with paper and pen or by means of a laptop, seem more reliable, and are particularly well-suited to reporting behaviours that can compromise the respondent in some way, such as intimate or painful feelings or illegal behaviours [25], [26]. In particular this could be attributable to the absence of a “direct witness,” which ensures the respondent's anonymity [27]. This data collection mode also decreases the inhibitory effect of the immediate environment (in particular family, when the survey takes place within a household) [12], [13], [28]. Undoubtedly the choice of data collection mode cannot act as an absolute guarantee of quality. It requires strict compliance with a specific protocol (being alone in the room, preferably being questioned outside the home, etc.), to prevent the psychological or material environment in which the respondent is placed from undermining the advantages expected from this method.

Several studies on drug use have shown that self-administered pen-and-paper questionnaires and CASI provide similar results, both in the United States [29]–[31] or in Europe [32], [33]. However, many studies argue for computer survey technology, especially for the A-CASI in adults [8], [34] and in adolescents [11], [35] because this mode is presented as the best way to explicitly provide confidentiality and anonymity to the respondent.

In several other studies on methodology for general population surveys, the telephone interview has been also presented as a compelling alternative, as it is often considered more anonymous than face-to-face interviews with an investigator for health surveys [36]–[38], for sensitive topics [39]–[41], and especially for drug use [6], [7], [19], [42]–[47]. Phone surveys are also known to considerably reduce costs compared with face-to-face interviews.

Nevertheless, some issues have received less consideration. Our main research question was the following: Is a telephone interview equivalent to A-CASI in collecting information on drug use? From another perspective, we were also interested in measuring potential gender or age effects in these results, because these variables can more easily be taken into account than others in a multimode survey design in order to select the most appropriate data collection mode for each respondent. Methodological studies rarely receive sufficient funding to establish such thorough comparisons. But in the study of drug use, where accuracy is always questionable and where there are strong gender and age effects, these issues are nevertheless worth investigating. Age and gender differences in reported substance use prevalence rates can lead to different representations of those behaviors; these differences are probably also often linked, however, to differing propensities to declare such behaviors.

It is generally accepted that self-administration leads to more honest responses about sensitive behaviours or attitudes than interviewer administration [26]. We thus expected that the A-CASI, which is often presented as the most private data collection mode, would correspondingly encourage more frequent reporting of these behaviours. The aim of this paper is to assess the differences in responses on drug use behaviours in A-CASI and telephone surveys. To do so we compared two large random cross-sectional surveys conducted six months apart in France in 2005 and containing the same questions on drug use. We performed multivariate analyses stratified by age and gender, with special attention paid to young subjects (18–29 years old), among whom illicit substance use is most prevalent.

Because there are several differences between HB and LEHS in sampling procedures, sampling frame, interviewers and questionnaire, such a comparison can not be stricto sensu considered as an experimental design. These differences are possible sources of variation between HB and LEHS results. However, several researchers were implicated in the preparation of both surveys, and this methodological project was designed prior to the surveys. Although everything was done in the survey design to favour reliability of this comparison, it has not been possible to definitely untangle all these different effects. These points are discussed. Nevertheless, a number of important studies on data collection mode effects have been conducted following the same procedure which enables to benefit from very large sample sizes compared to experimental designs [4], [7], [8], [10], [12], [13].