This study utilized GSV virtual audit methods to visually verify the physical locations of website-derived cannabis retailers across 180 counties in 13 jurisdictions in 2015, 2016, 2017, and 2018. We found that for most states sampled (with the exception of California), visual verification was high for the time-period contemporaneous to data collection, but decreased further back in time. Collectively, our findings suggest that GSV virtual audit methods may be appropriate for visually verifying retailers contemporaneously in many US jurisdictions legalizing cannabis sale, but are not appropriate for reconstructing historic retailer lists.

Visual verification was 74% or greater in all but two states in 2018. This finding is comparable with a study verifying cannabis retailer locations by telephone across 26 states, which found that 79% of website-derived cannabis retailers had correct addresses listed on websites (Williams et al., 2023). When examining visual verification within and between legislative periods, we found visual verification varied within the ballot era and between the ballot era and both the late and early legislative periods. The variation observed within the ballot period and between legislative periods when compared to the ballot era was likely driven by California (where only 41% of retailers were verified), as compared to 75% in Washington and 84% in Oregon. In California, the state with the greatest number of extracted retailers, the percent of website-derived retailers verified as cannabis retailers was consistently lower than other states sampled, across all time periods. These results were comparable with past studies using online and telephone verification of website-derived cannabis retailers in California, in which 39% (Cao et al., 2020) to 64% (Pedersen et al., 2018, 2020; Williams et al., 2023) of retailers in California were verified. Retailers in California are regulated by local jurisdictions resulting in a range of local regulations (Freisthler & Gruenewald, 2014; Freisthler et al., 2013; Pacula & Smart, 2017; Thomas & Freisthler, 2017). Shifts in local regulations may promote frequent retailer turnover (Fuller, 2019; Thomas & Freisthler, 2017). Because websites may not remove retailers once closed, this may result in overestimation of retailers in online data sources. Additionally, low verification may be due in part to the existence of a robust unregulated retail market (Herrington, 2019; Thomas & Freisthler, 2017). Unlicensed retailers may be less likely to utilize distinct storefront advertising required for visual verification (Herrington, 2019; Thomas & Freisthler, 2017). Clarity of cannabis symbology on retailer storefronts is a criterion for using Google virtual audit methods to verify the physical locations of retailers (Clarke et al., 2010; Nesoff et al., 2020; Rundle et al., 2011; Rzotkiewicz et al., 2018). Collectively, our findings, placed in context of other work demonstrating low verification of California retailer lists across multiple verification methods (e.g., phone, online) underscores the need for a multimodal, robust approach to validating retailer lists (Cao et al., 2020; Pedersen et al., 2018, 2020; Williams et al., 2023).

Our ability to visually verify the physical locations of retailers decreased the further we went back in time. Reasons for low visual verification varied. Low visual verification in 2017 was driven by small sample size (several states sampled had newly enacted medical laws and small retailer counts). In contrast, low visual verification in 2015 was largely driven by missing archived images. Past studies also identified infrequent historical image capture as a limitation of GSV (Rzotkiewicz et al., 2018). GSV was launched in 2014. Utilizing GSV to verify website-derived retailers may be a more useful tool in more recent years as GSV continues to expand. Of importance, all of the counties included in this study were in urban areas due to the parent study design. Past work has found that GSV may not be appropriate for identifying business establishments in rural settings (Crawford et al., 2019); area urbanicity should be considered when selecting verification methods. In instances where GSV images were available, lower visual verification may have been due in part to rapid turnover (e.g., closure) of retailers or new market growth over time (Amiri et al., 2019; Freisthler et al., 2016; Thomas & Freisthler, 2017). GSV may be a useful tool to track the existence of a single retailer over time (e.g., to identify whether a retailer was open at a given address for multiple years) in instances where images are available, but is not appropriate for reconstructing historic retailer lists more broadly, as it will not identify historic retailers at addresses not included in the source dataset. The feasibility and utility of using alternative approaches, such as machine learning methods, to construct historic lists should be explored (Doiron et al., 2022).

Our study is among the first to verify the physical locations of website-derived cannabis retailers using GSV virtual audit methods across multiple states and years. This study used robust data cleaning methods to remove duplicate or incomplete observations prior to visual verification. Other studies have also emphasized the importance of excluding incomplete cannabis retailer addresses extracted from websites (Shi et al., 2018; Williams et al., 2023). However, several limitations should be noted. Findings may not be generalizable to jurisdictions not included in this study, due to variability in state cannabis markets or area urbanicity. We utilized a single approach to verifying cannabis retailers extracted from websites. This method was closely tied to the availability and quality of archived GSV images. Therefore, cannabis retailers may have been missed due to poor image resolution in GSV. Additionally, variation in laws governing storefront advertising of cannabis products (e.g., limiting storefront signs to only include the name of the business, not allowing depictions of a plant or leaf on storefronts), may have hindered the virtual audit verification process (Leafly, 2022).

In summary, this study used GSV virtual audit methods to systematically and retrospectively validate cannabis retailer locations extracted from online sources across multiple states in the US, addressing a critical methodological gap in enhancing precision and reproducibility of cannabis retailer measures in cannabis research. Cannabis retailer lists obtained from websites require robust data cleaning to remove extraneous observations (e.g., items missing address information). We found that using GSV virtual audit methods may be appropriate for visually verifying retailers contemporaneously, but is not appropriate for reconstructing historic retailer lists. The appropriateness of GSV as a verification tool depends on several factors, including GSV coverage in the geographic area(s) under consideration, urbanicity, and local regulations governing storefront advertising by cannabis retailers. Some states, such as California, which has demonstrated low verification of retailers across multiple validation methods (e.g., phone, online) may require the use of multimodal verification methods. High-quality longitudinal data across multiple states are needed to inform evidence-based policy approaches to cannabis legalization. Future studies should assess the feasibility and validity of using GSV in conjunction with novel approaches, such as machine learning methods, and as well as other approaches previously used for verifying medical and adult-use cannabis retailers (e.g., calling retailers, in-person site visits).