In this cross-sectional study of 100 ED encounters, we found that LLMs could generate accurate discharge summaries, but were liable to hallucination and omission of clinically relevant information. Overall, GPT-4-generated summaries contained fewer errors than GPT-3.5-turbo summaries across all three domains, with 10%, 42% and 47% of summaries containing inaccuracies, hallucinations and omissions, respectively. GPT-4-generated summaries were also shorter and more readable than those generated by GPT-3.5-turbo, with an average Flesch-Kincaid Grade Level of 10.

The improved performance of GPT-4 compared to GPT-3.5-turbo aligns with prior literature which has shown superior GPT-4 performance across both medical and non-medical tasks.^22–24 Moreover, the fact that GPT-4 summaries contained a lower number of omissions than GPT-3.5-turbo, whilst summarizing the same information in fewer words, suggests increased summary concision that may be welcomed by primary care physicians and others on the receiving end of the transition of care.²⁵

Although only 33% of summaries generated by GPT-4 were entirely error-free across all domains, a more detailed review of the subtypes of error demonstrated that a majority of hallucinations either related to information redacted in the original note as part of our institution’s de-identification process or resulted from GPT-4 hallucinating follow-up instructions and/or return precautions. In the latter instance, such follow-up instructions were often appropriate for the patient’s care (as if they were derived from a standard set of precautions associated with the patient’s final diagnosis), but because they had not been explicitly mentioned in the original EM provider’s note, they were classified as hallucinations in accordance with our pre-specified protocol. After excluding these specific types of errors post-hoc, the proportion of GPT-4 generated summaries considered error-free across all domains increased by 14%, reaching 47% error-free across the three domains.

Meanwhile, there were notable differences in initial inter-reviewer agreement between error type prior to consensus agreement, with 91.9% agreement on the presence of clinical omissions compared to 95.8% and 93.6% agreement for inaccuracies and hallucinations, respectively. This reflects the subjective nature of classifying clinical omissions, where the inclusion of different clinical details may depend on the preference of the discharging clinician. It is possible that, with either dedicated prompt engineering or the addition of few-shot examples during future prompting, clinician-specific preferences of what information ought to be included in each discharge summary may be incorporated to address this.

There is a paucity of existing literature examining the performance of LLMs when generating discharge summaries, either in the Emergency Department or inpatient hospital setting. This is concerning given reports of the recent deployment of ambient artificial intelligence (AI) scribes at a large healthcare organisation.¹⁹ In that study, 35 example patient transcripts and encounter summaries generated by the AI scribe were rated using a modified version of the Physician Documentation Quality Instrument, with an average score of 48/50 achieved.^19,26 However, a quantitative analysis of the number and type of errors present was not reported. Meanwhile, a separate study of neurology inpatient encounters showed that Bidirectional Encoder Representations from Transformers (BERT) and Bidirectional and Auto-Regressive Transformers (BART) models could be used to generate summaries which met the standard of care in 62% of cases, but acknowledged that future work should count the number and type of hallucinations in automated summaries.¹⁸

Since clinicians will ultimately be responsible for auditing and modifying clinical documentation produced by LLMs, gaining a thorough understanding of potential error sources in this documentation is critically important. Without a thorough understanding of where errors may occur, there’s a risk that errors made by LLMs could be overlooked, potentially harming patient care.²⁷ Additionally, the increased workload on clinicians to meticulously audit the discharge summary could lead to worsening burnout, potentially negating the benefits of using this technology. Our findings suggest that the location of errors within a GPT-generated discharge summary may vary based on the type of error: inaccuracies and hallucinations are most commonly found within the Plan sections of GPT-generated discharge summaries, while the Physical Examination and History of Presenting Complaint sections should be checked closely for clinical omissions. Future studies should evaluate the application of LLMs themselves to identify instances of inaccuracy, hallucination and clinical omission errors within LLM-generated clinical documents when compared to the original source documents, allowing clinicians to audit and amend areas that are subject to discordance.

This study has several limitations. First, in this study only the initial EM clinician note was summarized. While this note typically contains the patient’s clinical history, physical examination findings, results of investigations performed and overall plan, other pertinent information that is found in notes from other providers, such as physical or occupational therapist recommendations and specialty consult advice, may not have been included in the discharge summary. Future work should evaluate the performance of LLMs in the more complex task of multi-document summarization before deployment to EDs can be considered. Second, due to the time and labor-intensive process of manual expert review, we included 100 randomly selected ED encounters in our sample, which may limit generalizability across different types of patient demographics and presenting symptoms. Notably, our randomly selected sample predominantly consisted of White, Asian or Black/African American patients, with limited representation of other minority groups. As LLM performance continues to be evaluated across different medical tasks, racial and gender bias assessments of these tools must be performed prior to their integration into clinical care.²⁸ Third, GPT model performance may improve with further iterations of prompt engineering and/or in-context learning. For instance, in comparing GPT-3.5-turbo to GPT-4, there was an enhancement in summarization capabilities across all domains evaluated, including over ED discharge summary length. Fourth, we did not directly compare the GPT-generated discharge summaries with the actual clinician-generated discharge summaries for these encounters. It is possible that important information might have been missing, or inaccurately reported, in the clinician-generated discharge summaries as well.