Comprehensive aortic stenosis characterization using multi-view deep learning
1Department of Medicine, Division of Cardiovascular Medicine, Stanford University School of Medicine, Stanford, CA
2Stanford Cardiovascular Institute, Stanford University, Stanford, CA
3Division of Research, Kaiser Permanente Northern California, Pleasanton, CA
4Department of Cardiology, Smidt Heart Institute, Cedars-Sinai Medical Center, Los Angeles, CA
5Department of Bioengineering, University of California, Los Angeles, CA
6Department of Medicine, University of California, San Francisco, CA
7San Francisco VA Medical Center, San Francisco, CA
8Kaiser Permanente San Francisco Medical Center, San Francisco, CA, USA
9Department of Research and Evaluation, Kaiser Permanente Southern California, Pasadena, CA
10Department of Cardiology, Kaiser Permanente Los Angeles Medical Center, Los Angeles, CA
11Department of Computer Science, Stanford University, Stanford, CA
12Department of Cardiology, Kaiser Permanente Santa Clara Medical Center, Santa Clara, CA
*Corresponding author: Dr. David Ouyang David.ouyang@kp.org 4480 Hacienda Drive Pleasanton, CA 94588Abstract
Background and Aims
Accurate assessment of aortic stenosis (AS) requires integration of both structural and functional information characterized by visual traits as well as quantitation of gradients. Existing artificial intelligence (AI) models utilize solely either structural or functional information.
Methods
We developed EchoNet-AS, an open-source end-to-end integrated approach combining video based convolutional neural networks to assess valve motion as well as segmentation models to automate the measurement of aortic valve peak velocity to classify AS severity.
Results
EchoNet-AS was trained on 210,193 images from 16,076 studies from Kaiser Permanente Northern California (KPNC) and validated on 1,589 held-out test studies and a temporally distinct cohort of 19,206 studies. The final model was also externally validated on 2,415 studies from Stanford Healthcare (SHC) and 9,038 studies from Cedars-Sinai Medical Center (CSMC). Combining assessments from multiple echocardiographic videos and Doppler measurements, EchoNet-AS achieved excellent discrimination of severe AS with AUC 0.964 [95% CI: 0.952 – 0.973] in the KPNC held-out cohort and 0.985 [0.981 – 0.988] in the temporally distinct cohort, which was superior to models using single views or only Doppler measurements. The performance was consistently robust in distinct external cohorts with an AUC 0.985 [0.975 – 0.992] at SHC and 0.989 [0.986 – 0.992] at CSMC.
Conclusions
EchoNet-AS synthesizes information from both B-mode videos and Doppler images to accurately assess AS severity. Its strong performance generalizes robustly to external validation cohorts and shows potential as an automated clinical decision support tool.
Article notes
Competing Interest Statement
DO reports consulting or honoraria for lectures from EchoIQ, Ultromics, Pfizer, InVision, the Korean Society of Echocardiography, and the Japanese Society of Echocardiography. Y.S. has received honoraria for consulting from m3.com, Inc and InVision.
Funding Statement
This work is funded by National Institutes of Health (R00HL157421, R01HL173487 and R01HL173526) and Alexion.
Introduction
Aortic stenosis (AS) is the most common heart valve disease, especially among older adults, and is associated with high rate of mortality and morbidity1–3. Echocardiography is the primary modality for assessing AS severity, and with the advent of transcatheter aortic valve replacement (TAVR), accurate evaluation of AS has become increasingly critical for identifying patients who require intervention, tailoring monitoring frequency for disease progression, and guiding treatment decisions. However, the current diagnostic standard of using human-interpreted transthoracic echocardiograms is limited by interobserver variability and the subjective nature of video interpretation. This is particularly problematic at the clinically important moderate–severe boundary, where misclassification can lead to delayed TAVR referral, unnecessary repeat echocardiograms, or inconsistent treatment recommendations across sites.
Recent advances in deep learning methods have enabled severity prediction of valvular heart disease from echocardiography images or videos4–9. For AS, prior studies have demonstrated that artificial intelligence (AI) models can accurately detect severe AS using single-view videos10,11, and others have proposed a commercialized software to automate Doppler-derived measurements to assess AS severity12,13. However, in clinical practice, it is important to review multiple videos and account for physiologic factors (especially in more challenging diagnostic scenarios such as with low-flow low-gradient AS) to holistically understand the severity of AS14,15.
Given that existing approaches do not leverage the complementary information available from multiple videos and Doppler measurements in a comprehensive echocardiographic assessment, we hypothesized that an integrated framework combining all relevant echocardiographic information could improve the accuracy and clinical reliability of AS assessment. We developed an open-source deep learning framework, EchoNet-AS, which integrates information from multiple echocardiographic views and Doppler measurements to predict the severity of AS (Figure. 1) and validated the framework across multiple institutions to demonstrate its generalizability.
Methods
Study population, imaging and data source
In this multicenter retrospective study, we used transthoracic echocardiography (TTE) data from three different medical centers: Kaiser Permanente Northern California (KPNC), Stanford Healthcare (SHC), and Cedars-Sinai Medical Center (CSMC) (Figure 2). A total of 608,086 TTE videos from 48,324 studies, obtained from 45,852 unique patients, were used for model development and testing. In KPNC, 17,665 studies conducted between January 1, 2022 and December 31, 2023 were sampled, with cases without AS downsampled to mitigate class imbalance during model training. This cohort was split into a derivation dataset (= training + validation; 16,076 TTE studies from 15,213 patients) and a held-out test cohort (1,589 TTE studies from 1,583 patients), ensuring no patient overlap between the subsets. The final derivation dataset comprised of 16,076 studies with 210,193 echocardiographic videos.
To assess temporal generalizability, we additionally collected 20,271 TTE studies conducted in 2024, of which 19,206 studies from 18,452 unique patients remained after excluding overlapping individuals from the derivation dataset. All the videos were recorded from adult participants on Philips (Amsterdam, The Netherlands) EPIQ 7C, EPIQ CVx, Affiniti 70C or Affiniti CVx machines. The acquisition of images followed a comprehensive and standardized protocol in-line with current recommendations14,16,17. The images were stored as Digital Imaging and Communications in Medicine (DICOM) files, and these files underwent pre-processing to ensure consistency and extraneous details beyond the ultrasound section were removed, and metadata was de-identified.
Additionally, our model was evaluated on two distinct external cohorts from separate healthcare systems. A total of 2,415 studies from 2,304 patients conducted in 2018 were extracted from Stanford Healthcare (SHC; Stanford California, USA) database. In addition, 9,038 TTEs from 8,300 patients conducted between 2013 and 2022 at the Cedars-Sinai Medical Center (CSMC; Los Angeles, California, USA) were extracted. The SHC and CSMC echocardiography lab uses Philips EPIQ 7 or IE33 machines and follows a similar standardized comprehensive protocol. AS severity was extracted from finalized echo report. Finally, to compare our model with the previously published model to detect severe AS, we extracted the same 5,572 TTE studies from CSMC, and then followed the approach of a prior study with a cohort of 4,226 studies for head-to-head comparison (CSMC replication cohort)11. For all cohorts, AS severity was categorized as ‘no, ‘mild’, mild to moderate’, ‘moderate’, ‘moderate to severe’ or ‘severe’. The reference labels used for training and evaluation were derived directly from the finalized physician echocardiography reports. No additional adjudication or consensus review was performed as part of this study, reflecting real-world reporting practice.
AI model development
We developed a two-stage framework for AS assessment, termed EchoNet-AS, consisting of video-based deep learning predictions from both B-mode/Doppler videos and ensemble integration with the model outputs and Doppler measurements (Figure 1). In the first stage, six separate video-based convolutional neural networks (CNNs) based on the ResNet R(2+1)D architecture18, which has demonstrated strong performance in echocardiography-related video tasks4. These models were trained to classify AS severity from six standard echocardiographic views: parasternal long axis (PLAX), parasternal short axis (PSAX), apical five/three chamber views (AP), PLAX color Doppler, PSAX color Doppler, and AP color Doppler. The input videos were processed to mask the embedded texts. The resulting videos were subsampled by one in every two frames to yield 16 frames, and resized to 112 × 112 pixel. Cross-entropy loss was employed as the loss function, and early stopping after 10 epoch based on validation area under the receiver operating characteristic curve (AUC) were employed to prevent overfitting. All the models were trained using the Pytorch 2.5.0 and PyTorch Lightning 2.5.1 19 deep learning framework with the Rectified Adam Schedule Free optimizer 20 with a batch size of 64 and mixed-precision training (bfloat16) 21. Training was performed on NVIDIA L40S GPUs, and the model weights from the epoch achieving the highest validation AUC were selected as the final models. In parallel, we utilized a previously described segmentation model based on DeepLabv3 (EchoNet-Measurement) that enables automated estimation of aortic peak velocity, achieving high accuracy with a mean absolute error of 10.4 cm/s in external validation22.
The second stage of the framework consisted of an ensemble model. The outputs from six view-specific networks—each generating a probability vector across the six AS severity classes— together with the peak aortic velocity measurement were used as input features for a machine learning ensemble implemented with the HistGradientBoostingClassifier in scikit-learn23. The final output is the probabilities of four AS severity classes (‘no’, ‘mild’, ‘moderate’ or ‘severe’). When multiple videos were available for the same view, the model outputs were averaged prior to integration. The ensemble classifier is robust to missing values, thereby allowing inclusion of studies that did not contain all six views. In parallel, we also constructed ensembles using only the three B-mode models as well as ensembles incorporating all six models (the three B-mode models and the three color-Doppler models). Hyperparameter tuning was performed using the validation dataset.
Statistical analysis
The EchoNet-AS framework was evaluated across four independent test cohorts. For the description of the baseline cohort characteristics, categorical variables are presented as counts and percentages, while continuous variables are expressed as mean and standard deviation. We calculated the area under the receiver operating characteristic curve (AUC) as the main outcome. Positive predictive value (PPV), negative predictive value (NPV), recall (sensitivity), specificity and F1 score were calculated for clinically significant AS, which was defined as greater than moderate AS and severe AS. Intermediate AS categories (e.g. “mild to moderate”) were upgraded to the upper categories (e.g. “moderate”). All 95% confidence intervals were calculated from 10,000 bootstrapping replicates. Subgroup analysis was conducted in KPNC temporally distinct cohort and CSMC cohort to assess model performance in patients with different ranges of age, sex, body mass index (BMI), left ventricular ejection fraction (LVEF), associated comorbidities (at least moderate aortic regurgitation, mitral regurgitation, mitral stenosis, and tricuspid regurgitation), aortic valve morphology, and other clinical characteristics. To understand which regions of the echocardiographic videos contributed most for AI model assessment, we employed smooth grad saliency mapping24. For ensemble model, we calculated the feature importance of input features by permutation importance function in scikit-learn and summarized them by echocardiographic view level. Analyses were performed using Python (version 3.12.3) and R (version 4.4.3)
Results
Study population
The KPNC cohort patients (mean age: 73.1 ± 14.2 years in derivation cohort, 72.3 ± 14.8 years in held-out test cohort, 74.0 ± 14.2 years in temporally distinct test cohort) were older than external validation cohorts (65.7 ± 16.2 years in the SHC cohort and 65,7 ± 16.8 years in the CSMC cohort). The KPNC derivation and held-out test cohort were enriched for mild/moderate/severe AS patients to mitigate class imbalance during model training (see Method). The proportion of male participants, racial/ethnic groups, the mean left ventricular ejection fraction, and major comorbidities were similar across datasets. A detailed description of the characteristics of the cohort is shown in Table 1.
Model performance
In the initial analysis of the performance of individual models for each standard echocardiographic view, the PLAX view model demonstrated the best performance in detecting severe AS (sTable 1). While performance varied across individual views, with the PLAX and apical views generally achieving the highest AUCs, the ensemble models that integrated multiple views consistently outperformed any single-view model across all cohorts (Table 2). In the KPNC temporally distinct test data, the ensemble of 3 B-mode views (PLAX, PSAX and AP) achieved an AUC of 0.976 [0.973 – 0.980] in the temporally distinct test data and the ensemble of all 6 video models achieved an AUC of 0.979 [0.975 – 0.983]. By incorporating automated Doppler measurements, the full EchoNet-AS framework demonstrated the best performance, with an AUC of 0.955 [0.946 - 0.964] for moderate or severe AS and 0.964 [0.952-0.973] for severe AS in the KPNC held-out test cohort and 0.983 [0.980 - 0.985] for moderate or severe AS and 0.985 [0.981-0.988] for severe AS in the KPNC temporally distinct cohort. The Echonet-AS generalized well in the external SHC (AUC, moderate or severe AS 0.989 [0.984 – 0.993], severe AS 0.985 [0.975 – 0.992]) and CSMC cohort (AUC; moderate or severe AS 0.978 [0.974 – 0.982]; severe AS 0.989 [0.986 – 0.992]) (Table 2, Figure 3).
Across all cohorts, positive predictive value (PPV) ranged from 0.731 to 0.848, negative predictive value (NPV) from 0.909 to 0.991, recall from 0.622 to 0.911, specificity from 0.917 to 0.996, and F1-score from 0.701 to 0.848, indicating consistent performance across diverse evaluation metrics (Table 3). Finally, we compared EchoNet-AS with the published severe AS detection model. Since the model weights were not publicly available, we extracted the same 5,572 TTE studies from the CSMC database and downsampled severe-AS cases as described in the published paper11, yielding 4,226 studies including 65 severe AS cases. In this CSMC replicated cohort, our ensemble of multi-view and doppler measurement approach demonstrated better severe AS discrimination performance with AUC of 0.976 [0.970 – 0.979] (sTable 2). In the CSMC replication cohort (N=4,226), our single view model showed comparable performance compared to prior approaches (AUC 0.956 [0.940 – 0.969]) to detect severe AS, while the ensemble approach with EchoNet-AS yielded superior overall performance with AUC of 0.972 [0.963 – 0.979].
Subgroup analysis
Analyzing patient subgroups further supported the generalizability of our framework. In the KPNC temporally distinct cohort, older patients (moderate or severe AS/severe AS: 0.983 [95% CI, 0.980–0.985] / 0.985 [0.981–0.988]), males (0.992 [0.984–0.998] / 0.998 [0.995–0.999]), females (0.980 [0.977–0.983] / 0.984 [0.978–0.988]), patients with obesity (BMI > 35: 0.985 [0.983–0.988] / 0.987 [0.983–0.990]), and those with reduced LVEF ≤ 50% (0.984 [0.982–0.987] / 0.986 [0.981–0.989]) all demonstrated high performance patients (0.881 [0.781–0.956] / 0.838 [0.622–0.995]). Performance was also robust in those with concomitant valvular disease such as AR, MR, TR, and MS. Slightly lower performance was observed in the small bicuspid aortic valve subgroup. Similar results were observed in the CSMC cohort: older patients (0.972 [0.966–0.976] / 0.986 [0.982–0.990]), males (0.979 [0.974–0.983] / 0.988 [0.983–0.992]), females (0.978 [0.971–0.983] / 0.991 [0.987–0.994]), obese patients (0.986 [0.978–0.993] / 0.994 [0.987–0.998]), and those with LVEF ≤ 50% (0.963 [0.952–0.973] / 0.977 [0.966–0.987]) consistently achieved high AUCs. Across both cohorts, severe AS detection generally outperformed moderate or severe AS detection, suggesting greater discriminative ability for severe cases (Table 4).
Explainability analysis
Saliency map analysis revealed that the view-specific models predominantly highlighted the aortic valve and surrounding regions on two-dimensional images, as well as the corresponding color Doppler signals around the aortic valve (Figure 4A). In the ensemble model without peak velocity, feature importance analysis demonstrated that features derived from the PLAX view contributed most strongly to the final prediction, followed by the apical and PSAX views. When peak aortic velocity was incorporated as an additional input, it became the single most influential feature, surpassing all view-specific model outputs, which was consistent with our expectations given its established role in assessing AS severity (Figure 4B, C).
Discussion
In this study, we developed and validated EchoNet-AS, a novel automated deep learning– based framework that estimates AS severity by incorporating video-based information from multiple echocardiographic views along with automated Doppler measurements of peak aortic jet velocity. Our multi-view approach demonstrated excellent performance (AUC ranging from 0.964 to 0.989 for discriminating severe AS, and 0.955 – 0.978 for discriminating at least moderate AS), while maintaining favorable PPV/NPV, recall, specificity, and F1-scores across 4 independent test cohorts. This model consistently outperformed single-view models, underscoring the complementary value of multi-view assessment and quantitative hemodynamics. Although the absolute AUC gains compared to single-view models may appear modest, even small improvements in discrimination can translate into fewer misclassified patients in practice, potentially reducing unnecessary repeat echocardiograms, shortening delays in referral for valve intervention, and improving consistency across sites.
In recent years, AI have been applied to echocardiography across a range of tasks, including assessment of left ventricular systolic dysfunction25,26, detection of left ventricular hypertrophy and its various subtypes27, detection of valvular regurgitation4–6,9. For AS specifically, previous work has demonstrated the feasibility of predicting severe AS from a single PLAX view; for example, Dai et al. developed deep learning models to discriminate whether the mean transvalvular pressure is greater than 40 mmHg and AVA is smaller than 1 cm2 from PLAX videos, achieving an AUC of 0.79 in the held-out test dataset10. Holste et al. developed a deep learning model to predict severe AS from PLAX images using 17570 videos and demonstrated an AUC of 0.952 [0.941 – 0.963] on external validation datasets. Our PLAX view model showed similar or greater performance (AUC 0.919 – 0.972), and our work builds upon these studies by expanding the size of training dataset (210,193 videos from 16,076 studies), and systematically comparing single-view, multi-view, and view-plus-measurement strategies. EchoNet-AS generalized across an KPNC held-out cohort, a temporally distinct cohort from a later year, and two external cohort (SHC and CSMC) with differing patient and echocardiogram acquisition characteristics. We also observed some variability across sites and between tasks (severe AS vs moderate or severe AS). This may reflect differences in case mix and documentation practices, and the fact that the moderate–severe boundary is inherently more challenging and noisier, whereas clearly severe cases tend to be more distinct. This breadth of validation reduces the risk that performance reflects idiosyncrasies of a single site or period. Moreover, performance remained robust across subgroups defined by age, sex, and comorbidities, suggesting resilience to common clinical heterogeneity. Performance was somewhat lower in patients with bicuspid valves, likely reflecting different leaflet morphology and eccentric jets, but was similar across patients with preserved versus reduced LVEF and in those with concomitant valve disease. These findings suggest the framework is broadly applicable across typical clinical scenarios.
Consistent with current clinical practice, incorporating multiple views and Doppler information are necessary to assess AS severity. Multiple views capture complementary information about aortic valve morphology, jet orientation, and surrounding structures. Ensembling across views mitigates view-specific limitations (e.g., suboptimal probe angle, foreshortening, or shadowing) and stabilizes predictions. The importance of multi-view assessment is emphasized in the literature14,28. Beyond images, peak aortic jet velocity is a key quantitative marker of AS severity; incorporating an automated estimate anchors the model in physiological signal. The resulting framework therefore fuses structural (B-mode), flow (color Doppler), and quantitative (peak velocity) evidence—mirroring expert practice—and delivers consistent gains over image-only ensembles. By ensembling predictions from all six view-specific models, we achieved further improvements in classification performance, underscoring the complementary value of multi-view integration. The inclusion of quantitative measurement— specifically, peak aortic velocity—further enhanced predictive performance, which is consistent with its established clinical importance in AS grading. By flagging studies with severe or moderate AS, the system could help reduce the risk of missed severe AS on echocardiography and facilitate more timely intervention. In instances where the model’s prediction disagrees with the physician’s initial interpretation, a secondary review of the echocardiographic videos and reassessment of severity grading could be undertaken, which may further enhance diagnostic precision.
There are several limitations of our study. First, the ground truth for AS severity was based on clinical reports rather than a dedicated core lab analysis, which could introduce some variability in the reference standard. However, this approach reflects real-world clinical practice, and all studies were interpreted by board-certified cardiologists following standardized guidelines. Second, while peak velocity is a major AS severity anchor, future work could incorporate additional quantitative measures (e.g., mean gradient or aortic valve area14) and other clinical parameters. Finally, this study lacks the prospective validation; future work through prospective randomized and integrated into routine workflow26,29,30, is required to establish effectiveness and the impact on patient care.
Conclusions
In summary, we propose EchoNet-AS, an open source, video-based framework that integrates multi-view B-modes, Doppler color flow and an automated Doppler peak-velocity measurement to estimate AS severity. Validated across four independent cohorts, EchoNet-AS demonstrated robust performance, particularly for detecting severe AS, and could serve as a valuable clinical decision support tool for AS assessment in routine clinical practice.
Declarations
Disclosure of Interest
DO reports consulting or honoraria for lectures from EchoIQ, Ultromics, Pfizer, InVision, the Korean Society of Echocardiography, and the Japanese Society of Echocardiography.
Data availability
The dataset of echocardiography videos and reports used to train EchoNet-AS is not available for public sharing, given the restrictions in our institutional review board approval.
Funding
This work is funded by National Institutes of Health (R00HL157421, R01HL173487 and R01HL173526) and Alexion.
Code availability
Our code and model weights are available at https://github.com/echonet/AS.
Ethical approval
This study received approval from the Institutional Review Boards of KPNC, SHC and CSMC. Informed consent was waived due to the study’s use of secondary analysis of existing data.