Artificial Intelligence in Healthcare: 2024 Year in Review
1BrainXAI ReSearch, BrainX LLC
*Corresponding author: Piyush Mathur (pmathurmd@gmail.com)ABSTRACT
Background
With over a thousand FDA-approved artificial intelligence/machine learning-enabled medical devices, research and publications is maturing from focusing on the development and internal validation of models to the external validation of models and implementation trials. Foundation models, especially Large Language Models, have spurred additional aspects of AI research related to healthcare, especially with the use of text-based data to address healthcare education and administrative tasks related to patient care.
Methods
We performed a PubMed search using the terms “machine learning” or “artificial intelligence” and “2024,” restricted to English language and human subject research on January 1, 2025. Utilizing a deep learning-based approach, we assessed the maturity of publications. Following this, we manually annotated the healthcare specialty, data utilized, and models employed for the identified mature articles. Subsequently, empirical data analysis was performed to elucidate trends and statistics. We also performed a detailed analysis of the distribution of foundation model-based publications amongst the healthcare specialties.
Results
For the year 2024, the PubMed search yielded 28,180 articles, of which 1,693 were classified as mature using a BERT model. Following exclusions, 1,551 articles were selected for the final data analysis. Amongst these, the highest number of articles in each specialty originated from Imaging (407), Head and Neck (127), and General (122). The analysis of data types revealed that image data (903 [57.0%]) was still the predominant data type, but the use of text data (525 [33.1%]) had substantially increased. Additionally, we also found that LLMs (479) and AI General (448) category models have overtaken deep learning models (372) in healthcare AI research. For LLM-related publications, we are seeing increasing trends in research related to healthcare education and administrative tasks.
Conclusion
With the introduction of foundation models, healthcare research trends are changing. The adoption of LLMs and text data types amongst various healthcare specialties, especially for education and administrative tasks, is unlocking new potential for AI applications in healthcare.
Article notes
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
This study did not receive any funding
Summary of Updates:
INTRODUCTION
Artificial intelligence has seen a substantial change in research and publication trends since the introduction of ChatGPT [1]. We saw a similar shift in research trends related to AI in healthcare in 2023 [2]. In the past, deep learning models using image-based data were the primary method used in research and publications related to AI in healthcare [3]. These publications were predominantly in the fields of Imaging (Radiology), Oncology, Pathology, and Ophthalmology, amongst others [3]. With the introduction of Large Language Models(LLMs), we saw a shift towards using text-based data and a keen interest in trialing these models by every healthcare specialty.
This review aims to provide a comprehensive evaluation of publications related to AI applications in healthcare in 2024 and a comparative analysis of the trends for the past few years. We continue to use maturity-based assessments of the publications to sift through high volumes of research, and use a deep learning model, which performs with a high degree of accuracy [4]. Once again, we provide a manual analysis of data and model type to provide a detailed overview of the selected publications. With the growth in the use of generative AI, we continue to provide quantitative analysis and a review of publications related to the newly evolving field of generative AI in healthcare.
METHODOLOGY
We performed a PubMed search (Figure 1) using the terms, “machine learning” or “artificial intelligence” and “2024”, restricted to English language and human subject research as of December 31, 2024 on January 1st 2025. This search resulted in an initial pool of 28,180 publications. Our methodology has remained consistent since 2019, which allows for comparative analysis of publications for each medical specialty, year over year [2,3]
We performed a qualitative evaluation of the publications’ maturity with additional details related to the type of data used and the type of models developed across the healthcare spectrum. We used a Bidirectional Encoder Representation from Transformer (BERT)-based maturity classification model that was pre-trained and validated on manually labeled data for ‘Mature’ and ‘Not Mature’ publications [4], to assess the level of maturity of each publication. The level of maturity was determined by the ability of the publication to answer the question: “Does the output of the proposed model have a direct, actionable impact on patient care by providing information to healthcare providers or automated systems?” Systematic reviews were excluded from the count of “mature” publications as they do not independently address the above question, as the models are variable in the systematic reviews [4].
Identified publications were manually reviewed and 116 publications were excluded from the 1612 mature ones. Most of these were related to robotic surgeries or non-human studies. Further mature publications were classified based on the healthcare specialty. The General category contains many of the publications related to general AI topics that were not specialty-specific, for instance drug development-related publications. Review articles such as systematic reviews or scoping reviews were separately classified and removed prior to performing further data and model-type analysis. Education and administrative aspects of healthcare-related publications are classified as a separate specialty to provide focused reviews for those groups of publications.
Lastly, similar to prior years, we manually annotated specific details from the remaining mature publications, such as data type & model type [2]. Data type was classified manually into four categories of data: image, text, tabular, and voice. Few publications used more than one type of data, for which credit was given to each of the data categories. Model type was also manually curated from abstracts into eight different categories: Deep Learning (DL), Machine Learning (ML), AI General, Statistical, Natural Language Processing (NLP), Large Language Model (LLM), Large Vision Model (FM-LVM), and Multimodal Model (FM-MM). Those publications where the model type was not described in the title or the abstract were placed in the “AI General” class. Foundation models were separately classified as Large Language Model (LLM), Large Vision Model (FM-LVM), and Multimodal Model (FM-MM). Many of these publications were related to validation studies related to proprietary AI models. Similarly, studies using statistical analysis to analyze surveys or performance of AI models were included in the statistical class. Due to the significant increase in LLM publications in 2024, we report and provide an analysis of only mature publications using foundation models including LLMs. This is different from last year where we did a separate search for LLM in healthcare publications and provided an analysis.[2].
RESULTS
The number of publications continues to increase as depicted by the number of articles identified by our standardized search methodology, reaching 28,180 in the year 2024. (Figure 2A). Qualitatively assessed for maturity, the number of publications curated through this method also saw a significant increase from the prior year (n = 1693) [2] (Figure 2B).
The distribution of mature articles across specialties was not evenly distributed, led by Imaging with 407 publications, followed by Head & Neck (includes dental) with 127, and General with 122 publications in the top 3. We have observed a significant number of publications dedicated to Education that primarily use Large Language Models. (Table 1).
Overall, Imaging continues to be the specialty with the highest number of mature articles, followed by Head & Neck, and General. Image-based specialties such as Gastroenterology, Oncology, Pathology, Ophthalmology & Orthopedics continue to dominate in the quantity of mature publications. Education, which saw no mature publications until 2021 as a separate specialty category, had substantial mature publications primarily related to the trials of LLM in these areas. In contrast, specialties such as Anesthesiology, Pulmonary, Genetics, Nephrology, and Psychiatry had fewer mature publications, with most of them using tabular or text data.
We evaluated the distribution of AI models used across all publications. LLMs have, for the first time, emerged as the most prominent with 479 publications, followed by AI General with 448 publications, DL (Deep Learning) with 372, ML (Machine Learning) with 205, NLP (Natural Language Processing) with 12, LVM (Large Vision Model) with 4, and statistical approaches with 15 publications (Table 2). For the first time, we have seen the use of multimodal foundation models (FM-MM) in healthcare publications (25 publications). We also found the use of different model types in various combinations, ML and DL (7 publications), ML and NLP (2 publications), and DL and NLP (1 publication).
We analyzed the distribution of data types across all mature publications, of which image data remains the dominant type in 903 publications. Text data has seen a substantial increase due to the substantial increase in research using LLMs, with its use in 525 publications this year. Tabular data was utilized in 149 publications, and audio data in only 8 publications (Figure 4). It is important to note that 45 publications used more than one type of data, for which credit was given to each of the data categories.
Foundation model-related publications have seen a substantial increase, including those using LLMs, multimodal models, and large vision models. Education (89) and administrative (32) applications of LLM research have seen substantial research interest and publications. Surgical specialties such as Head and Neck (52), Surgery (48), and Orthopedics are also among the top healthcare specialties researching the use of LLM building on their experiences from the use of deep learning methods and early adoption of AI research. Oncology (29) and Imaging (26), which have dominated AI-based research in the past many years, are included in the top few specialties trialing foundation models, with Imaging especially experimenting with multimodal models. Specialties such as Gastroenterology (5), Pathology (10), and Cardiology (8) which have also led in AI-based research and publications for the past many years, had fewer LLM-based publications, Anesthesiology (3), Nephrology (4), and Pulmonary (1) as healthcare specialties, similar to overall AI trends have an opportunity to learn to increase the levels of research on large language models (Figure 5).
DISCUSSION
AI in healthcare publications continues to see sustained increases in both the quantity 28,180) and quality (1793 [6.0%] mature) of publications. We observed a decrease in the number of exclusions, both due to errors related to PubMed search results and based on the number of scoping or systematic reviews. In 2024, once again, LLM-based publications continue to increase substantially across all healthcare specialties. Multimodal foundation models for the first time were experimented with, moving beyond LLMs and thereby bringing into focus many of the data types used in combination. Head & Neck (includes dental) publications experimenting with both deep learning techniques on image data and the use of LLMs saw a substantial increase relative to other publications. A substantial increase in text data for research is also due to increased focus on LLM-based research, a model type that, for the first time, had the highest number of publications. The use of proprietary models, noted as AI General, for trials is exciting to see and demonstrates the maturity of the field, moving from the development of models and internal validations to external validation trials.
Image-based AI applications in healthcare specialties such as Head & Neck, Gastroenterology, Ophthalmology, Oncology, Surgery, Head & Neck, and General categories continue to dominate in the number of mature publications (Table 1). These healthcare specialties, which have been leading in research and publications related to AI in healthcare, have also seen significant adoption of foundation models for experimentation. Another interesting trend is in the research related to the application of AI, especially LLMs, in areas of healthcare such as education and administrative tasks (Figure 5). Anesthesiology, Critical Care, Nephrology, and Rehabilitation continue to have low volumes of publications as they lack image-based data for research and have probably not developed resources for training, research and development similar to the leading healthcare subspecialties(Table 1). Not unexpectedly, we now observe a significant drop in publications related to COVID-19, which during the pandemic years had to be defined as a specialty of its own.
Maturity amongst Imaging as a healthcare specialty is reflected in external validations, and multicenter trials of many of the proprietary commercial AI models, moving beyond internal validations [5,6]. Many of their publications also focus on the impact of the implementation of these models both on the radiologists’ workflow and patient perceptions of their application [7,8]. Similarly, many of the publications in Gastroenterology focused on endoscopic images, are focused on trials of commercially available and device-implemented AI models undergoing multicenter trials [9–12]. Automation of Echocardiography, for ventricular function assessment, is a dominant area of study in Cardiology as a healthcare specialty [13,14]. Surgical publications, including those for Head and Neck, include a significant number of dentistry-focused publications, and have focused on the interpretation of images for decision-making [15–17]. Oncology-based publications feature the use of AI in various cancer screenings, such as for breast, cervical, and prostate cancer [18–20]. Additionally, many of the oncology publications focused on radiotherapy planning using cancer images [21]. Many other specialties, such as Ophthalmology and Neurology, are advancing from early phases of AI model development and trials to external validation studies to study real-world outcomes of their implementation [22,23]. Democratizing key healthcare clinician skill sets is an important aspect of value-based implementation of AI and is demonstrated in the use of estimation of gestational age from blind ultrasound sweeps, making care possibly available to patients with limited access to healthcare [24].
For the first time, we see LLMs and the AI General categories as the dominant categories of models in research, overtaking deep learning models. However, many of the proprietary models that have not been described in detail in the title and abstract of the publications amongst the AI General category might be utilizing deep learning models. The use of proprietary models for external validations also demonstrates and validates the maturity of AI applications being researched in healthcare, with over a thousand FDA-approved AI/ML devices [25]. There continues to be a significant increase in the utilization of text data was also observed, which may be attributed to the rapid advancement of large language models (LLMs). This advancement was evident in the distribution of model types across all papers, with LLMs being utilized in (479 [30.7 %]) of the analyzed publications. As predicted in last year’s review for the first time, we have also witnessed research related to multimodal foundation models that combine a variety of data types in healthcare [2]. We anticipate the use of these models to only grow with the use of combined data types and agent-based research in the near future.
Research related to the use of foundation models has seen exponential growth over the last two years. While LLMs have been the dominant models, large vision models and multimodal models that have been recently introduced are also being studied for application in healthcare [26–28]. These trials have a dominant focus on educational and administrative aspects of healthcare across many specialties. Publications related to the comparison between clinician experts’ performance on proficiency tests such as board examinations remain significant [29–31]. The use of LLMs for scholarly activity, framing of questions for trainee examinations, evaluation of student performance, and research applications such as the conduct of peer review has also gained significant interest [32–35]. Addressing patient queries, providing readable and understandable healthcare content for informed consent, and patient education are additional areas of interest across most of the healthcare specialties [36,37]. Unburdening the clinicians by addressing the administrative aspects of healthcare using LLMs is another area of research where studies related to the automation of discharge summaries, responses to patient messages, and generation of billing codes continue to grow [38–40]. GPT continues to be the dominant model for comparison, but many other models are also being trialed and compared for their accuracy and efficacy.
We acknowledge certain limitations of our analysis based on our search or analysis methodology. The search was limited to PubMed and with the restrictions mentioned in the methodology section. It is possible that some of the significant publications might have been missed due to our methodology. In prior years, we have manually excluded many publications from the PubMed search and classified publications into various select healthcare specialities. Due to the volume of publications (28,180), it was not possible to perform any manual evaluation of all the publications. We have tried various methods, including BERT and LLMs, for automated classification, including fine-tuning on prior data, but the results were not satisfactory. Analysis of mature publications provides a more focused review, and a consistent methodology also allows us to provide year-over-year comparative evaluations [2,3]. We have also evaluated alternate approaches to learn maturity, specifically focusing on methodologies derived from scholarly literature and journal metrics, such as no. of citations, journal impact factors, and journal H-index etc. However, it is important to note that using these metrics to evaluate maturity can be misleading when applied to the analysis of articles from the most recent year due to the unpredictable dynamic trends of these metrics [41–43]. The current BERT-based approach, despite its limitations, remains content-aware, reproducible even with changing factors such as article citation count and journal impact factor, and fine-tunable with new data.
CONCLUSION
The maturity of the research publications related to AI in healthcare is evident from the increasing external validation trials of proprietary models across many of the leading healthcare specialties. The application of foundation models in healthcare-related publications is transforming the capabilities of AI with the increasing use of text and combined modalities of data, especially in non-clinical areas such as education and administrative aspects of healthcare. The quantity and, more importantly, the maturing quality of AI research in healthcare continue to grow across all healthcare specialties.
DATA AVAILABILITY
Data is available upon request via email to the corresponding author or by contacting us through our website, BrainXAI Research (https://www.brainxai.com/research). Additional related publications and datasets related to AI in Healthcare can be accessed from BrainX Community website(https://www.brainxai.org).