Changes in public perception of AI in healthcare after exposure to ChatGPT
1Steno Diabetes Center Aarhus, Aarhus University Hospital, Denmark
2Department of Public Health, Aarhus University, Denmark
*Corresponding author; email: andaas@rm.dkAbstract
Background
Artificial intelligence (AI) is expected to become an integral part of healthcare services, and the widespread adoption of AI tools in all areas of life is making AI accessible to the general public. Public perception of the benefits and risks of AI in healthcare is key to large-scale acceptance and implementation, and is increasingly influenced by first-hand experiences of AI. The aim of this study was to assess how exposure to ChatGPT changed public perception of AI in healthcare.
Methods
We used baseline and follow-up data from 5,899 survey participants, who reported their perception of AI in 2022 and 2024, and ChatGPT use in 2024. Administrative and healthcare data from nationwide Danish registers was used for weighting and adjustment. Multinomial multivariate logistic regression was used to model how exposure to ChatGPT use affected changes in perception of AI.
Results
At baseline (before ChatGPT’s launch) 2,236 individuals (37%) were unsure of the benefits and risks of AI in healthcare, 2,384 (40%) perceived net benefits, 1,083 (18%) perceived benefits and risks as equal, and 196 (3.3%) perceived net risks. At follow-up, 1,195 individuals (20%) had been exposed to ChatGPT use, which was associated with higher odds of changing perception of AI to benefits (OR 3.21 [95% CI: 2.34-4.40]) among individuals who were unsure at baseline, and lower odds of changing to uncertainty from more defined baseline perceptions (from benefits (OR 0.32 [0.24-0.42]), equal (OR 0.47 [0.32-0.69]) and risks (OR 0.27 [0.08-0.98])).
Conclusion
Exposure to ChatGPT was associated with a change towards positive perception of benefits and risks of AI in healthcare among individuals who were uncertain prior to exposure, and individuals with more defined perceptions of AI were less likely to become uncertain after exposure to ChatGPT.
Article notes
Competing Interest Statement
AAI, JRS, LB, and AH are employed at Steno Diabetes Center Aarhus which is partly funded by an unrestricted grant from the Novo Nordisk Foundation (NNF17SA0031230). AAI and AH are supported by a Data Science Emerging Investigator grant (NNF22OC0076725) by the Novo Nordisk Foundation. The funders had no role in the design of the study.
Clinical Protocols
Funding Statement
This study did not receive any funding.
Background
The implementation of artificial intelligence (AI) technology in healthcare is expected to increase efficiency, improve care quality, relieve the growing burden on healthcare systems,1 and empower patients’ access to knowledge.2 The practical implementation of this technology is growing at a rapid pace, as evidenced by the number of annual Food and Drug Administration (FDA) approvals for AI-enabled medical devices, which increased from around five to a thousand in the last decade.3,4 With the prospect of AI being widely implemented in healthcare settings, exposing patients and the general public to these tools, it is important to understand how first-hand experiences with AI tools influence perceptions of benefits and risks of applying AI in healthcare - particularly since a large part of the public is uncertain or undecided about their perception of AI.5–7
Previously, one of the challenges for the public to form a clear opinion on the application of AI was that these technologies were invisible elements of daily life.8 In recent years, AI has become a visible, accessible and practical tool that the public increasingly uses and interacts with in everyday life - particularly since the launch of ChatGPT in November 2022 and its rapid adoption, sometimes referred to as the “large language model revolution”.9 With this change, the public’s perception of AI is likely to be shaped by practical experiences from first-hand use rather than intangible or theoretical considerations. Patients were quick to adopt ChatGPT and other general-purpose generative AI tools for guidance on personal healthcare conditions, highlighting the close relationship between AI and healthcare.10 While the public’s overall perception of AI in healthcare has been studied in cross-sectional surveys across the world,5,6,11 the evidence on the impact of first-hand exposure to AI and longitudinal changes in perception is sparse.
During discussions with members of the user panel of patients and relatives at Steno Diabetes Center Aarhus, a diabetes outpatient centre in Denmark,12 about the results of a previous survey of perception of AI in healthcare,5 some members noted how impressed they were with ChatGPT and remarked that they had changed their perception after their first use. Inspired by this, we hypothesized that ChatGPT exposure would most strongly impact the large group of the public (one-third of survey respondents) who were unsure about AI’s benefits and risks. This study aims to provide evidence on the impact of being exposed to first-hand use of AI tools, such as ChatGPT, on the perception of AI in healthcare. Specifically, we investigate changes in public perception of AI in healthcare between 2022 and 2024, and assess the impact of exposure to ChatGPT on change in perception from the baseline perception reported before ChatGPT’s release.
Materials and Methods
Data collection
In this study, we collected data on perception of AI and ChatGPT use, and combined it with administrative data from nationwide Danish registers.
Survey data: Health in Central Denmark (HICD) is a digital and postal questionnaire survey initially conducted in 2020 on all inhabitants of Central Denmark Region aged 18 to 74 years identified as prevalent diabetes cases in register data on 31 December 2018, plus an equally-sized group of people without diabetes (matched to diabetes cases by sex, age, and municipality)13. Among respondents in 2020 that volunteered to be contacted for further studies, a baseline survey on perception of AI was conducted in the spring of 2022,14 and these individuals were contacted again in 2024 as part of the next wave of the HICD survey, which provided follow-up data for this study.15
Perceived risks and benefits of AI in healthcare were assessed in baseline (2022) and follow-up (2024) surveys. Participants were asked, “Do you think the benefits of using artificial intelligence in the healthcare sector outweigh the risks?” with four answers: “Benefits outweigh risks” (“Benefits”), “Benefits equal risks” (“Equal”), “Risks outweigh benefits” (“Risks”), and “Don’t know” (the 2024 questionnaire contained two “don’t know”-responses options that were merged for this study).15 In the 2022 questionnaire, participants who reported having never heard of AI were not subsequently asked about their perception, but were classified as “Don’t know”. Information on exposure to ChatGPT use was reported in 2024 with the question “Have you used ChatGPT?” with the following answer options: “Yes, I use it often”, “Yes, I have tried it”, “No, but I have heard about it” or “No, I have not heard about it”. In the analyses, this variable was included as binary: “ChatGPT exposed” and “Unexposed”, by merging the two former and latter categories, respectively.
Administrative and healthcare data covering the entire Danish population can be linked at the individual level.16 Information on age, sex, and education level (defined by the total length of attained of education) was obtained from the Danish Civil Registration System17 and the Danish Population Education Register.18 Diabetes status was defined as prevalent “Type 1 diabetes” (T1D), “Type 2 diabetes” (T2D), or “No diabetes” at baseline according to the Open-Source Diabetes Classifier based on HbA1c measurements ≥48 mmol/mol (6.5%), hospital diagnoses of diabetes, diabetes-specific podiatrist services, and purchases of glucose-lowering drugs.19
Statistical Analysis
The distribution of study population characteristics was tabulated by exposure to ChatGPT use and reported as frequencies (%) or mean (SD). Alluvial plots and cross-tabulations were applied to visualize raw changes in perceptions of AI from baseline to follow-up between ChatGPT exposed and unexposed individuals. Multinomial logistic regression was used to model the association between ChatGPT use and the perception of AI use in healthcare in the follow-up survey, stratified by perception of AI in the baseline survey. To assess the change in perception from baseline, the reference category of the outcome differed between each strata to reflect the baseline perception in each group. The models were adjusted for age (continuous), sex, education level (“<10 years”, “10-15 years”, and “>15 years”) and diabetes status and applied stabilized propensity score weights based on the distribution of these covariates in the background population (capped at 95% percentile). Supplementary analyses include unweighted analyses and analyses using weights computed against the 90,854 invitees in the initial 2020 HICD survey (Figure S3 and S4), and overall analyses using unstratified models adjusted for baseline perception (Table S6). Statistical analyses were pre-registered20 and performed using R version 4.4.121 and packages for data processing,22 weighting,23 regression modelling,24 and visualizations.25,26
Results
The flow of participants from the background population into the study population is shown in Figure 1. Of 90,854 individuals invited to the initial HICD survey, 12,755 (14%) consented to be contacted for further studies, of whom 5,934 (47%) responded in both the 2022 and 2024 surveys. After excluding 35 individuals without information on education level, the study population consisted of 5,899 individuals, of whom 365 (6.2%) had T1D and 2,540 (43%) had T2D.
Overall characteristics of the study population and by ChatGPT exposure are shown in Table 1. At baseline, 2,236 of these 5,899 individuals (37%) were unsure of the benefits and risks of using AI in healthcare, while 2,384 (40%) perceived the benefits as greater than the risks, 1,083 (18%) perceived the benefits as equal to the risks, and 196 (3.3%) perceived the risks as outweighing the benefits.
At follow-up, 1,195 individuals (20%) had been exposed to ChatGPT use. These individuals tended to be younger (mean age 61 years vs. 66 years among unexposed), had a higher proportion of men (69% vs. 60%), lower proportion with diabetes (46% vs. 50%), and had a higher education (54% vs. 35%) than those unexposed. Only 155 (13%) of the 1,195 ChatGPT-exposed individuals reported using ChatGPT often, while 1,223 (26%) of the 4,704 unexposed individuals reported having never heard about ChatGPT.
At baseline, individuals who were subsequently exposed to ChatGPT were more likely to perceive benefits of AI in healthcare (62% vs. 35%) and less likely to be unsure (19% vs. 43%) than unexposed, while the two groups differed less in terms of proportions of individuals reporting “Equal” (17% vs. 19%) or “Risks” (2.7% vs. 3.5%). Study population characteristics weighted against the background population are available in Table S1, and characteristics stratified by baseline perception are available in Table S2 of the Supplementary Appendix.
In the follow-up survey, the overall proportion of individuals reporting “Benefits” decreased to 35%, while the proportion perceiving “Risks” doubled to 6.7% (p < 0.0001). Among individuals who reported “Don’t know” at baseline, 65 (29%) of the 226 who were subsequently exposed to ChatGPT use changed perception to “Benefits”, compared to 272 (14%) of the 2,010 unexposed individuals from this baseline group (Figure 2).
Figure 3 presents the odds ratios (ORs) for changing perception versus maintaining baseline perception at follow-up associated with exposure to ChatGPT use (e.g. an OR > 1 indicates a higher likelihood of change in perception among individuals exposed to ChatGPT compared to unexposed individuals) for each baseline perception group. Exposure to ChatGPT use was associated with higher odds of changing perception to “Benefits” (OR 3.21 [95% CI: 2.34-4.40]) or to “Equal” (OR 1.46 [1.01-2.11]) vs. maintaining the baseline perception in individuals who reported “Don’t know” at baseline. In the remaining baseline perception groups, exposure to ChatGPT use was associated with lower odds of changing perception to “Don’t know” vs. maintaining the baseline perception at follow-up among individuals reporting “Benefits” (OR 0.32 [0.24-0.42]), “Equal” (OR 0.47 [0.32-0.69]) or “Risks” (OR 0.27 [0.08-0.98]) at baseline. A full list of model coefficients is available in supplementary (Table S5).
In supplementary analyses (Figure S3 & S4) using unweighted models and models weighted to the characteristics of invitees to the initial HICD survey, the estimates for ChatGPT exposure were consistent with the main analysis. However, the estimates for change of perception from “Don’t know” at baseline towards “Risks” (OR 2.35 [1.45-3.81]), “Equal” (OR 1.84 [1.25-2.73]) and “Benefit” (OR 2.92 [2.04-4.17]) at follow-up differed in magnitude between the unweighted and the main analyses.
Some baseline characteristics were associated with change of perception at follow-up in the regression models (Table S5). Generally, shorter length of education, older age and female sex were associated with lower odds of changing baseline perception from “Don’t know” to a more defined perception at follow-up - especially to “Benefits”, and these groups were also more likely to change perception to “Don’t know” at follow-up from other baseline perceptions. In overall analyses adjusted for - rather than stratified by - baseline perception, exposure to ChatGPT use was associated with lower odds of reporting “Don’t know”, “Equal” or “Risks” than “Benefits” at follow-up (Table S6).
Discussion
In this study, we found that exposure to ChatGPT reduced uncertainty of perceived benefits and risks of AI applied in healthcare, regardless of perception of AI prior to ChatGPT exposure. Furthermore, we found that exposure to ChatGPT mainly resulted in a change towards a positive perception of AI in healthcare among individuals who were uncertain prior to this exposure. Overall, the perception of AI in healthcare became more negative between 2022 and 2024, with a substantial decrease in the proportion of individuals perceiving mainly benefits and more than a doubling of those perceiving mainly risks. While these trends were present in both the ChatGPT exposed and unexposed group, it was less pronounced among individuals exposed to ChatGPT.
Overall, exposure to ChatGPT use was associated with shifts toward more positive perceptions of AI compared to non-exposure. This was driven by shifts from uncertainty at baseline to positive perception at follow-up, without any evidence of changes toward more positive or skeptic perceptions in individuals with other baseline perceptions. Unweighted analyses showed a more diverse influence of ChatGPT exposure in the uncertain group, with higher likelihood of shifting towards both risks and benefits. The study population was older than the background population due to the HICD survey invites being sampled based on the age distribution of individuals with diabetes, and this difference between the weighted and unweighted analyses may be due to a different influence of ChatGPT use among older uncertain individuals than among younger.
Uptake of this technology in the public was limited, as only one-in-five survey respondents had tried ChatGPT (and one-in-five had not even heard of it). Uptake was particularly poor among people with shorter education, women, and older age groups - characteristics which were all associated with changes towards negative or uncertain perceptions of AI or maintaining these skeptic perceptions from baseline to follow-up. This finding adds to previous reports of more sceptical outlooks on AI among individuals with low digital familiarity in the United Kingdom (UK).27 Thus, improving digital literacy in these demographics may reduce uncertainty and skepticism regarding benefits and risks of AI in healthcare.
As current and future patients, the public is a key stakeholder in the the implementation of AI in the healthcare system, and their perception of the benefits and risks of the technology may be critical to large-scale implementation and adoption of this technology.28 Public perception of the technology remained largely positive, with only a small minority perceiving risks as outweighing benefits at follow-up, but the growing skepticism towards AI in healthcare may become a concern if this trend grows in the coming years. Our study suggests that exposure to AI tools like ChatGPT influenced public perception of AI and could help mitigate growing skepticism, particularly as nearly half of the ChatGPT-unexposed group was undecided at follow-up.
Our study demonstrates the potential for changing public perceptions of AI in healthcare by reducing uncertainty among individuals through exposure to AI tools like ChatGPT. However, negative experiences with poorly performing AI tools could shape perceptions in a negative direction, and evidence on this dynamic remains sparse (File S7). To our knowledge, this is the first study to report how exposure to an AI tool directly impacts public perceptions of AI in healthcare, as the few existing studies on changes in perception after exposure to AI were limited to specific groups, such as healthcare professionals or patients. For instance, emergency radiology professionals reported reduced uncertainty and increased perceived benefits of AI following its implementation,29 while urolithiasis patients exposed to ChatGPT-generated notes experienced a negative shift in attitudes.30 Thus, there is a lack of evidence to guide the implementation of AI in healthcare, which is seeing increasing uptake across areas of healthcare and data modalities, such as electronic health record analysis for clinical diagnostics and prediction,31 clinical image classification,32,33 and voice recordings of patients for out-of-hours cardiac arrest detection34 and ambient AI scribes.35
We observed a growing skepticism toward AI in healthcare from 2022 to 2024 in our study, similar to trends in the UK’s Public Attitudes to Data and AI Tracker Survey, which found that the public became more pessimistic about the impact of AI on society from 2021 to 2023, partly due to data and privacy concerns.27 These findings contrast a previous study based on data from social media, which found a positive trend in sentiment towards AI in medical imaging over time from 2019 to 202436. This inconsistency may be due to differences in methodology and settings, but may also reflect that the public’s perception of AI is subject to external factors such as media coverage of AI and digital technology. The sentiment of media coverage may change substantially over time, geography and media source. Within research on the medical knowledge of ChatGPT, several reports have praised it for matching or exceeding the quality of human doctors in medical licensing exams37–40 and for showing higher levels of empathy in written communication with patients.41 In contrast, other studies have highlighted how ChatGPT may perform significantly worse than human doctors in more complicated, free-text clinical case exams,42 and how minor nuances in the wording of inputs can lead to highly biased outputs.43 Depending on which of these messages gets picked up by the latest mainstream media coverage, the public perception may shift according to this, rather than the public’s first-hand experiences with AI.
The strengths of this study include the large study population, which allowed for stratified analyses to identify specific changes rather than an overall effect associated with exposure to ChatGPT use. As the baseline perception data was collected shortly before the launch of ChatGPT, this allowed us to closely study the changes in perception after exposure to ChatGPT. Finally, the population-wide register data enabled weighted analyses to increase the representativeness of results and reduce the risk of selection bias inherent in survey-based studies.
There are some limitations to the study. Due to the observational nature of the study, confounding cannot be ruled out. As we did not assess all potential confounders of change in perception of AI that may have been related to personal use of ChatGPT, e.g. exposure to other AI tools or awareness of positive or negative media reports on use of AI or digital technology in healthcare, we cannot rule out that such external factors biased our results. Also, while the item on perception of AI in healthcare was similar in the baseline and follow-up surveys, the remaining content of the questionnaires differed, which could bias responses.44 Another limitation of this study is that the population represented the general public, who likely use ChatGPT as a general-purpose tool rather than for healthcare-specific purposes. While diabetes status did not influence the results, the findings may not apply to all patient populations or clinical AI use cases. Public perception of healthcare-specific AI tools likely depends on the tool, its user experience, and its context - factors beyond the scope of this study.
Conclusion
Exposure to ChatGPT was associated with a change towards positive perception of benefits and risks of AI in healthcare among individuals who were uncertain prior to exposure. Individuals with more defined perceptions of AI prior to exposure to ChatGPT were not swayed towards positive nor negative perceptions, but were less likely to become uncertain after exposure to ChatGPT. Public acceptance of AI in healthcare is key to large-scale implementation of the technology, and while we found increasing skepticism towards AI in healthcare among the public over time, this study demonstrates the potential for changing the public perception of AI in healthcare through exposure of the public to AI tools.
Supporting information
Data Availability
The Health in Central Denmark data is hosted on remote servers at Statistics Denmark. The project is managed by a steering committee at Steno Diabetes Center Aarhus, Aarhus University Hospital. The steering committee encourages interested researchers to use this resource. More information is available on the website: https://www.stenoaarhus.dk/research/resources/health-in-central-denmark/