Large Language Models in Radiologic Numerical Tasks: A Thorough Evaluation and Error Analysis.

Ali Nowroozi, Masha Bondarenko, Adrian Serapio, Tician Schnitzler, Sukhmanjit S. Brar, Jae Ho Sohn

The purpose of this study was to investigate the performance of LLMs in radiology numerical tasks and perform a comprehensive error analysis. We defined six tasks: extracting (1) minimum T-score from DEXA report, (2) maximum common bile duct (CBD) diameter from ultrasound report, and (3) maximum lung nodule size from CT report, and judging (1) presence of a highly hypermetabolic region on a PET report, (2) whether…