Assessing Artificial Intelligence and Radiologist Performance in Musculoskeletal Fracture Detection: Multi-Reader, Multi-Case Study
| Autoři | |
|---|---|
| Rok publikování | 2025 |
| Druh | Stať ve sborníku |
| Konference | Proceedings of the 1st International Conference on AI in Medicine and Healthcare (AiMH' 2025) |
| Citace | |
| Popis | Fracture detection on musculoskeletal (MSK) radiographs is critical for both emergency and routine care, yet diagnostic errors remain common due to high workloads and limited radiological expertise. This study evaluates the diagnostic performance of an artificial intelligence (AI) system (Carebot AI Bones 1.8.10; Carebot s.r.o.) in detecting fractures on MSK X-rays, comparing its performance to six radiologists of varying experience levels in a blinded multi-reader, multi-case (MRMC) study. A total of 489 radiographs were retrospectively analyzed from routine clinical practice, with ground truth established for 448 images through consensus among three experienced radiologists. Diagnostic performance was assessed using sensitivity (Se), specificity (Sp), positive (PLR), and negative likelihood ratio (NLR), with statistical analysis including McNemar’s test and Holm’s method for multiple comparisons. The AI system achieved a Se of 0.921 (95% CI: 0.846–0.961) and Sp of 0.897 (0.861–0.924). Radiologists’ Se ranged from 0.663 to 0.933, and Sp ranged from 0.916 to 0.989. The AI demonstrated consistent high sensitivity across body parts, particularly for elbow and hand/wrist fractures, often exceeding radiologists’ performance. Specificity was slightly lower but acceptable, supporting AI’s potential as a complementary diagnostic tool. These findings highlight the clinical utility of AI in MSK fracture detection, particularly in settings with limited resources or high diagnostic workloads. Future research should validate these results in larger, multicentric studies to ensure broader generalizability and evaluate AI integration in real-world workflows. |