AI improves accuracy, agreement and efficiency of pathologists for Ki67 assessments in breast cancer.

Amanda Dy, Ngoc-Nhu Jennifer Nguyen, Julien Meyer, Melanie Dawe, Wei Shi, Dimitri Androutsos, Anthony Fyles, Fei-Fei Liu, Susan Done, April Khademi
Author Information
  1. Amanda Dy: Electrical, Computer, and Biomedical Engineering, Toronto Metropolitan University, Toronto, ON, Canada. amanda.dy@torontomu.ca.
  2. Ngoc-Nhu Jennifer Nguyen: Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, ON, Canada.
  3. Julien Meyer: School of Health Services Management, Toronto Metropolitan University, Toronto, ON, Canada.
  4. Melanie Dawe: Princess Margaret Cancer Centre, University Health Network, Toronto, ON, Canada.
  5. Wei Shi: Princess Margaret Cancer Centre, University Health Network, Toronto, ON, Canada.
  6. Dimitri Androutsos: Electrical, Computer, and Biomedical Engineering, Toronto Metropolitan University, Toronto, ON, Canada.
  7. Anthony Fyles: Princess Margaret Cancer Centre, University Health Network, Toronto, ON, Canada.
  8. Fei-Fei Liu: Princess Margaret Cancer Centre, University Health Network, Toronto, ON, Canada.
  9. Susan Done: Princess Margaret Cancer Centre, University Health Network, Toronto, ON, Canada.
  10. April Khademi: Electrical, Computer, and Biomedical Engineering, Toronto Metropolitan University, Toronto, ON, Canada.

Abstract

The Ki-67 proliferation index (PI) guides treatment decisions in breast cancer but suffers from poor inter-rater reproducibility. Although AI tools have been designed for Ki-67 assessment, their impact on pathologists' work remains understudied. 90 international pathologists were recruited to assess the Ki-67 PI of ten breast cancer tissue microarrays with and without AI. Accuracy, agreement, and turnaround time with and without AI were compared. Pathologists' perspectives on AI were collected. Using AI led to a significant decrease in PI error (2.1% with AI vs. 5.9% without AI, p < 0.001), better inter-rater agreement (ICC: 0.70 vs. 0.92; Krippendorff's α: 0.63 vs. 0.89; Fleiss' Kappa: 0.40 vs. 0.86), and an 11.9% overall median reduction in turnaround time. Most pathologists (84%) found the AI reliable. For Ki-67 assessments, 76% of respondents believed AI enhances accuracy, 82% said it improves consistency, and 83% trust it will improve efficiency. This study highlights AI's potential to standardize Ki-67 scoring, especially between 5 and 30% PI-a range with low PI agreement. This could pave the way for a universally accepted PI score to guide treatment decisions, emphasizing the promising role of AI integration into pathologist workflows.

References

  1. Curr Oncol. 2023 Mar 06;30(3):3079-3090 [PMID: 36975446]
  2. PLoS One. 2014 Nov 06;9(11):e112121 [PMID: 25375149]
  3. J Chiropr Med. 2016 Jun;15(2):155-63 [PMID: 27330520]
  4. JAMA Netw Open. 2019 May 3;2(5):e194337 [PMID: 31150073]
  5. N Engl J Med. 2023 Aug 17;389(7):612-619 [PMID: 37585627]
  6. Diagn Pathol. 2020 Jul 4;15(1):80 [PMID: 32622359]
  7. Arch Pathol Lab Med. 2022 May 1;147(5):568-576 [PMID: 35939795]
  8. Breast Cancer. 2015 Mar;22(2):129-34 [PMID: 23543399]
  9. Lancet Oncol. 2023 Jan;24(1):77-90 [PMID: 36493792]
  10. J Natl Cancer Inst. 2021 Jul 1;113(7):808-819 [PMID: 33369635]
  11. JAMA Netw Open. 2020 Nov 2;3(11):e2023267 [PMID: 33180129]
  12. Appl Immunohistochem Mol Morphol. 2022 Apr 1;30(4):237-245 [PMID: 35384873]
  13. Med Image Anal. 2016 Oct;33:170-175 [PMID: 27423409]
  14. Cancers (Basel). 2021 Sep 03;13(17): [PMID: 34503265]
  15. Histopathology. 2021 Aug;79(2):210-218 [PMID: 33590577]
  16. J Natl Cancer Inst. 2011 Nov 16;103(22):1656-64 [PMID: 21960707]
  17. Ann Oncol. 2021 Oct;32(10):1216-1235 [PMID: 34242744]
  18. Mod Pathol. 2023 Mar;36(3):100033 [PMID: 36931740]
  19. N Engl J Med. 2004 Sep 2;351(10):963-70 [PMID: 15342804]
  20. Br J Cancer. 2013 Mar 19;108(5):1195-208 [PMID: 23449362]
  21. J Clin Oncol. 2005 Oct 1;23(28):7212-20 [PMID: 16192605]
  22. PLoS One. 2022 Mar 25;17(3):e0265905 [PMID: 35333879]
  23. PLoS One. 2018 Oct 25;13(10):e0205387 [PMID: 30359393]
  24. Front Bioeng Biotechnol. 2019 Oct 01;7:226 [PMID: 31632956]
  25. J Natl Cancer Inst. 2013 Dec 18;105(24):1897-906 [PMID: 24203987]
  26. Cancers (Basel). 2020 Dec 22;13(1): [PMID: 33375043]
  27. Ann Oncol. 2021 Dec;32(12):1571-1581 [PMID: 34656740]
  28. Lancet Oncol. 2010 Feb;11(2):174-83 [PMID: 20152769]
  29. Breast Cancer Res Treat. 2015 Oct;153(3):477-91 [PMID: 26341751]
  30. Cancers (Basel). 2021 Mar 09;13(5): [PMID: 33803148]
  31. Histopathology. 2021 Oct;79(4):544-555 [PMID: 33840132]
  32. J Clin Oncol. 2020 Dec 1;38(34):3987-3998 [PMID: 32954927]
  33. J Clin Oncol. 2015 Jun 20;33(18):2035-40 [PMID: 25964246]
  34. Appl Immunohistochem Mol Morphol. 2021 Feb 1;29(2):105-111 [PMID: 32590453]
  35. Cancer Biol Med. 2016 Dec;13(4):496-504 [PMID: 28154782]

MeSH Term

Humans
Female
Breast Neoplasms
Ki-67 Antigen
Pathologists
Reproducibility of Results
Immunohistochemistry

Chemicals

Ki-67 Antigen

Word Cloud

Created with Highcharts 10.0.0AI0Ki-67PIagreementvsbreastcancerpathologistswithouttreatmentdecisionsinter-raterturnaroundtime59%assessmentsaccuracyimprovesefficiencyproliferationindexguidessufferspoorreproducibilityAlthoughtoolsdesignedassessmentimpactpathologists'workremainsunderstudied90internationalrecruitedassesstentissuemicroarraysAccuracycomparedPathologists'perspectivescollectedUsingledsignificantdecreaseerror21%p < 0001betterICC:7092Krippendorff'sα:6389Fleiss'Kappa:408611overallmedianreduction84%foundreliable76%respondentsbelievedenhances82%saidconsistency83%trustwillimprovestudyhighlightsAI'spotentialstandardizescoringespecially30%PI-arangelowpavewayuniversallyacceptedscoreguideemphasizingpromisingroleintegrationpathologistworkflowsKi67

Similar Articles

Cited By