Explainable machine learning and online calculators to predict heart failure mortality in intensive care units.

An-Tian Chen, Yuhui Zhang, Jian Zhang
Author Information
  1. An-Tian Chen: Department of Cardiology, Fuwai Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College/National Center for Cardiovascular Diseases, Beijing, China. ORCID
  2. Yuhui Zhang: Heart Failure Center, Fuwai Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College/National Center for Cardiovascular Diseases, Beijing, China.
  3. Jian Zhang: Heart Failure Center, Fuwai Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College/National Center for Cardiovascular Diseases, Beijing, China.

Abstract

AIMS: This study aims to develop explainable machine learning models and clinical tools for predicting mortality in patients in the intensive care unit (ICU) with heart failure (HF).
METHODS: Patients diagnosed with HF who experienced their first ICU stay lasting between 24 h and 28 days were selected from the Medical Information Mart for Intensive Care IV (MIMIC-IV) database. The primary outcome was all-cause mortality within 28 days. Data analysis was performed using Python and R, with feature selection conducted via least absolute shrinkage and selection operator (LASSO) regression. Fifteen models were evaluated, and the most effective model was rendered explainable through the Shapley additive explanations (SHAP) approach. A nomogram was developed based on logistic regression to facilitate interpretation. For external validation, the eICU database was utilized.
RESULTS: After selection, the study included 2343 records, with 1808 surviving and 535 deceased patients. The median age of the study population was 70.00, with ~3/5 males (60.31%). The median length of stay in the ICU was 6.00 days. The median age of the survival group was younger than the non-survival group (69.00 vs. 73.00), and non-survival patients spent longer time in the ICU. Seventy-five features were initially selected, including basic information, vital signs, laboratory tests, haemodynamics and oxygen status. LASSO regression determined the shrinkage parameter �� = 0.020, and 44 features were chosen for model construction. The linear discriminant analysis (LDA) model showed the best performance, and the accuracy reached 0.8354 in the training cohort and 0.8563 in the testing cohort. It showed satisfying area under the curve (AUC), recall, precision, F1 score, Cohen's kappa score and Matthew's correlation coefficient. The concordance index (c-index) reached 0.7972 in the training cohort and 0.8125 in the testing cohort. In external validation, the LDA model achieved approximately 0.9 in accuracy, precision, recall and F1 score, with an AUC of 0.79. Univariable analysis was performed in the training cohort. Features that differed significantly between the survival and non-survival groups were subjected to multiple logistic regression. The nomogram built on multiple logistic regression included 14 features and demonstrated excellent performance. The AUC of the nomogram is 0.852 in the training cohort, 0.855 in the internal validation cohort and 0.770 in the external validation cohort. The calibration curve showed good consistency.
CONCLUSIONS: The study developed an LDA and a nomogram model for predicting mortality in HF patients in the ICU. The SHAP approach was employed to elucidate the LDA model, enhancing its utility for clinicians. These models were made accessible online for clinical application.

Keywords

References

  1. Am J Public Health. 2021 Dec;111(12):2149-2156 [PMID: 34878854]
  2. Circulation. 2000 Jun 13;101(23):E215-20 [PMID: 10851218]
  3. JACC Heart Fail. 2021 Dec;9(12):904-915 [PMID: 34857174]
  4. Dis Markers. 2020 Nov 16;2020:8833637 [PMID: 33282007]
  5. J Clin Invest. 2013 Jan;123(1):46-52 [PMID: 23281409]
  6. N Engl J Med. 2021 Jan 14;384(2):105-116 [PMID: 33185990]
  7. Am J Cardiol. 2021 Jan 15;139:64-70 [PMID: 33115639]
  8. Front Immunol. 2022 Dec 14;13:1076121 [PMID: 36591285]
  9. Nat Rev Cardiol. 2012 Dec;9(12):717-33 [PMID: 23090087]
  10. North Clin Istanb. 2015 Dec 31;2(3):189-195 [PMID: 28058366]
  11. Stroke. 2019 May;50(5):1263-1265 [PMID: 30890116]
  12. Med Sci Sports Exerc. 2009 Jan;41(1):19-25 [PMID: 19092707]
  13. Circulation. 2015 Nov 17;132(20):1920-30 [PMID: 26572668]
  14. J Am Coll Cardiol. 2018 Jun 12;71(23):2668-2679 [PMID: 29880128]
  15. Sci Data. 2018 Sep 11;5:180178 [PMID: 30204154]
  16. Sci Rep. 2022 Jul 16;12(1):12183 [PMID: 35842444]
  17. Front Cardiovasc Med. 2022 Oct 12;9:994359 [PMID: 36312291]
  18. J Transl Int Med. 2023 Jul 5;11(2):138-144 [PMID: 38025953]
  19. Comput Biol Med. 2020 Jan;116:103580 [PMID: 32001013]
  20. Comput Biol Med. 2021 Feb;129:104151 [PMID: 33290932]
  21. J Transl Int Med. 2024 May 21;12(2):188-196 [PMID: 38978967]
  22. Circ Res. 2021 May 14;128(10):1421-1434 [PMID: 33983838]
  23. Eur J Heart Fail. 2015 Jun;17(6):601-11 [PMID: 26096207]
  24. JAMA. 2007 Jul 25;298(4):423-9 [PMID: 17652296]
  25. Am J Gastroenterol. 2020 Oct;115(10):1657-1668 [PMID: 32341266]
  26. J Am Coll Cardiol. 2008 Sep 2;52(10):818-27 [PMID: 18755344]
  27. J Am Coll Cardiol. 2013 May 14;61(19):1982-4 [PMID: 23500247]
  28. Circulation. 2021 Nov 2;144(18):1489-1499 [PMID: 34432985]
  29. Br J Anaesth. 2021 Jul;127(1):e25-e27 [PMID: 34052031]
  30. Crit Care. 2020 Jul 31;24(1):478 [PMID: 32736589]
  31. Circulation. 2018 Mar 20;137(12):e67-e492 [PMID: 29386200]
  32. J Transl Int Med. 2022 Jul 2;11(4):401-409 [PMID: 38130646]
  33. ESC Heart Fail. 2025 Feb;12(1):353-368 [PMID: 39300773]
  34. Minerva Anestesiol. 2023 Mar;89(3):166-174 [PMID: 36282224]
  35. JAMA. 2005 Oct 5;294(13):1664-70 [PMID: 16204666]
  36. Eur J Heart Fail. 2020 Aug;22(8):1342-1356 [PMID: 32483830]

Grants

  1. 7222143/Beijing Natural Science Foundation
  2. 2020-I2M-1-002/CAMS Innovation Fund for Medical Science
  3. 2022-GSP-GG-9/Fuwai Hospital, Chinese Academy of Medical Sciences

MeSH Term

Humans
Male
Heart Failure
Female
Intensive Care Units
Machine Learning
Aged
Hospital Mortality
Retrospective Studies
Prognosis
Nomograms
Risk Assessment
Survival Rate
Middle Aged

Word Cloud

Created with Highcharts 10.0.00cohortmodelICUregressionnomogramstudymortalitypatientsvalidationLDAtrainingmachinelearningmodelsheartfailureHFanalysisselectionlogisticexternalmedian00non-survivalfeaturesshowedAUCscoreexplainableclinicalpredictingintensivecarestay28 daysselecteddatabaseperformedshrinkageLASSOSHAPapproachdevelopedincludedagesurvivalgroupperformanceaccuracyreachedtestingcurverecallprecisionF1multipleonlineAIMS:aimsdeveloptoolsunitMETHODS:Patientsdiagnosedexperiencedfirstlasting24 hMedicalInformationMartIntensiveCareIVMIMIC-IVprimaryoutcomeall-causewithinDatausingPythonRfeatureconductedvialeastabsoluteoperatorFifteenevaluatedeffectiverenderedShapleyadditiveexplanationsbasedfacilitateinterpretationeICUutilizedRESULTS:2343records1808surviving535deceasedpopulation70~3/5males6031%length600 daysyounger69vs73spentlongertimeSeventy-fiveinitiallyincludingbasicinformationvitalsignslaboratorytestshaemodynamicsoxygenstatusdeterminedparameter�� = 002044chosenconstructionlineardiscriminantbest83548563satisfyingareaCohen'skappaMatthew'scorrelationcoefficientconcordanceindexc-index79728125achievedapproximately979UnivariableFeaturesdifferedsignificantlygroupssubjectedbuilt14demonstratedexcellent852855internal770calibrationgoodconsistencyCONCLUSIONS:employedelucidateenhancingutilitycliniciansmadeaccessibleapplicationExplainablecalculatorspredictunitspredictive

Similar Articles

Cited By