Use and Usefulness of Risk Prediction Tools in Urologic Surgery: Current State and Path Forward.
Elizabeth M Nazzal, Allison M Deal, Benjamin Borgert, Hillary Heiling, Antonia V Bennett, Susan Blalock, William Meeks, Raymond Fang, Randall Teal, Maihan B Vu, David Gotz, Matthew Nielsen, Alex H S Harris, Ethan Basch, Hung-Jui Tan
Author Information
Elizabeth M Nazzal: Department of Urology, School of Medicine, University of North Carolina, Chapel Hill, NC.
Allison M Deal: Lineberger Comprehensive Cancer Center, School of Medicine, University of North Carolina, Chapel Hill, NC.
Benjamin Borgert: Department of Urology, School of Medicine, University of North Carolina, Chapel Hill, NC.
Hillary Heiling: Lineberger Comprehensive Cancer Center, School of Medicine, University of North Carolina, Chapel Hill, NC.
Antonia V Bennett: Department of Health Policy & Management, Gillings School of Global Public Health, University of North Carolina, Chapel Hill, NC.
Susan Blalock: Pharmaceutical Outcomes & Policy, Eshelman School of Pharmacy, University of North Carolina, Chapel Hill, NC.
William Meeks: American Urological Association Data and Statistical Services.
Raymond Fang: American Urological Association Data and Statistical Services.
Randall Teal: Lineberger Comprehensive Cancer Center, School of Medicine, University of North Carolina, Chapel Hill, NC.
Maihan B Vu: Lineberger Comprehensive Cancer Center, School of Medicine, University of North Carolina, Chapel Hill, NC.
David Gotz: Lineberger Comprehensive Cancer Center, School of Medicine, University of North Carolina, Chapel Hill, NC.
Matthew Nielsen: Department of Urology, School of Medicine, University of North Carolina, Chapel Hill, NC.
Alex H S Harris: Department of Surgery, School of Medicine, Stanford University, Palo Alto, CA.
Ethan Basch: Lineberger Comprehensive Cancer Center, School of Medicine, University of North Carolina, Chapel Hill, NC.
Hung-Jui Tan: Department of Urology, School of Medicine, University of North Carolina, Chapel Hill, NC.
INTRODUCTION: While the enthusiasm for artificial intelligence (AI) to enhance surgical decision-making continues to grow, the preceding advance of risk prediction tools (RPTs) has had limited impact to date. To help inform the development of AI-powered tools, we evaluated the role of RPTs and prevailing attitudes among urologists. METHODS: We conducted a national mixed methods study using a sequential explanatory design. Through the 2019 AUA Census, we surveyed urologists on RPT use, helpfulness, and trust. Based on responses, we interviewed 25 participants on RPTs, risk evaluation, and surgical decision-making. Coding-based thematic analysis was applied and integrated with survey findings. RESULTS: Among 2,081 urologic surgeons (weighted sample 12,366), 30.4% (95% CI 28.2-32.6%) routinely used RPTs and 34.3% (95% CI 31.9-36.6%) found them helpful while 47.0% (95% CI 44.6-49.5%) generally trusted their own assessment over RPT-generated estimates. More years in practice was negatively associated with RPT use, helpfulness, and trust (p<0.001). Qualitatively, participants described relying on their intuition for surgical risks and benefit and employing gist-based approximations rather than numerical information, which RPTs provide. RPT helpfulness centered on risk/benefit confirmation, calibration, and communication, but methodological (e.g., individual vs. group estimates, missing variables) and operational (e.g., ease of use, clinical workflow) challenges limit greater RPT use. CONCLUSIONS: Despite their wide availability, RPTs remain limited in their use and helpfulness. This reflects both the intuitive nature of surgical decision-making and implementation challenges. For AI to reach its promise and improve surgical care and outcomes, both types of barriers will need to be addressed.
