Minimum labelling requirements for dermatology artificial intelligence-based Software as Medical Device (SaMD): A consensus statement. - National Genomics Data Center (CNCB-NGDC)

Advanced Search

Minimum labelling requirements for dermatology artificial intelligence-based Software as Medical Device (SaMD): A consensus statement.

��sa Ingvar, Ayooluwatomiwa Oloruntoba, Maithili Sashindranath, Robert Miller, H Peter Soyer, Pascale Guitera, Tony Caccetta, Stephen Shumack, Lisa Abbott, Chris Arnold, Craig Lawn, Alison Button-Sloan, Monika Janda, Victoria Mar

Author Information

��sa Ingvar: Victorian Melanoma Service, Alfred Health, Melbourne, Victoria, Australia. ORCID
Ayooluwatomiwa Oloruntoba: School of Public Health and Preventive Medicine, Monash University, Melbourne, Victoria, Australia.
Maithili Sashindranath: School of Public Health and Preventive Medicine, Monash University, Melbourne, Victoria, Australia.
Robert Miller: Australasian College of Dermatologists, Sydney, Australia.
H Peter Soyer: Australasian College of Dermatologists, Sydney, Australia. ORCID
Pascale Guitera: Australasian College of Dermatologists, Sydney, Australia.
Tony Caccetta: Australasian College of Dermatologists, Sydney, Australia.
Stephen Shumack: Australasian College of Dermatologists, Sydney, Australia. ORCID
Lisa Abbott: Australasian College of Dermatologists, Sydney, Australia. ORCID
Chris Arnold: BioGrid Australia Ltd, Melbourne, Australia.
Craig Lawn: Melanoma Institute Australia, The University of Sydney, Sydney, New South Wales, Australia.
Alison Button-Sloan: Australian Melanoma Consumer Alliance, Melbourne, Victoria, Australia.
Monika Janda: Australasian College of Dermatologists, Sydney, Australia.
Victoria Mar: Victorian Melanoma Service, Alfred Health, Melbourne, Victoria, Australia.

PMID: 38419186 DOI: 10.1111/ajd.14222

BACKGROUND/OBJECTIVES: Artificial intelligence (AI) holds remarkable potential to improve care delivery in dermatology. End users (health professionals and general public) of AI-based Software as Medical Devices (SaMD) require relevant labelling information to ensure that these devices can be used appropriately. Currently, there are no clear minimum labelling requirements for dermatology AI-based SaMDs.
METHODS: Common labelling recommendations for AI-based SaMD identified in a recent literature review were evaluated by an Australian expert panel in digital health and dermatology via a modified Delphi consensus process. A nine-point Likert scale was used to indicate importance of 10 items, and voting was conducted to determine the specific characteristics to include for some items. Consensus was achieved when more than 75% of the experts agreed that inclusion of information was necessary.
RESULTS: There was robust consensus supporting inclusion of all proposed items as minimum labelling requirements; indication for use, intended user, training and test data sets, algorithm design, image processing techniques, clinical validation, performance metrics, limitations, updates and adverse events. Nearly all suggested characteristics of the labelling items received endorsement, except for some characteristics related to performance metrics. Moreover, there was consensus that uniform labelling criteria should apply across all AI categories and risk classes set out by the Therapeutic Goods Administration.
CONCLUSIONS: This study provides critical evidence for setting labelling standards by the Therapeutic Goods Administration to safeguard patients, health professionals, consumers, industry, and regulatory bodies from AI-based dermatology SaMDs that do not currently provide adequate information about how they were developed and tested.

Delphi consensus artificial intelligence dermatology labelling medical device

Rajpurkar P, Lungren MP. The current and future state of AI interpretation of medical images. N Engl J Med. 2023;388(21):1981��1990.
Kelly BS, Judge C, Bollard SM, Clifford SM, Healy GM, Aziz A, et al. Radiology artificial intelligence: a systematic review and evaluation of methods (RAISE). Eur Radiol. 2022;32(11):7998��8007.
Daneshjou R, Smith MP, Sun MD, Rotemberg V, Zou J. Lack of transparency and potential bias in artificial intelligence data sets and algorithms: a scoping review. JAMA Dermatol. 2021;157(11):1362��1369.
Caffery LJ, Janda M, Miller R, Abbott LM, Arnold C, Caccetta T, et al. Informing a position statement on the use of artificial intelligence in dermatology in Australia. Australas J Dermatol. 2023;64(1):e11��e20.
Maron RC, Haggenmuller S, von Kalle C, Utikal JS, Meier F, Gellrich FF, et al. Robustness of convolutional neural networks in recognition of pigmented skin lesions. Eur J Cancer. 2021;145:81��91.
Katragadda C, Finnane A, Soyer HP, Marghoob AA, Halpern A, Malvehy J, et al. Technique standards for skin lesion imaging: a Delphi consensus statement. JAMA Dermatol. 2017;153(2):207��213.
Daneshjou R, Barata C, Betz��Stablein B, Celebi ME, Codella N, Combalia M, et al. Checklist for evaluation of image��based artificial intelligence reports in dermatology: CLEAR derm consensus guidelines from the International Skin Imaging Collaboration Artificial Intelligence Working Group. JAMA Dermatol. 2022;158(1):90��96.
Radiologists TRAaNZCo. The RANZCR position paper on the regulation of artificial intelligence in medicine. 2022.
Dalkey NC. The delphi method: an experimental study of group opinion. Santa Monica, CA: RAND Corporation; 1969.
Snoswell CL, Caffery LJ, Whitty JA, Soyer HP, Gordon LG. Cost��effectiveness of skin cancer referral and consultation using teledermoscopy in Australia. JAMA Dermatol. 2018;154(6):694��700.
Department of Health TGA, Australian Government, editor. Regulatory changes for software based medical devices. Canberra, Australia: Department of Health TGA, Australian Government; 2021, editor.
Brereton TA, Malik MM, Lifson M, Greenwood JD, Peterson KJ, Overgaard SM. The role of artificial intelligence model documentation in translational science: scoping review. Interact J Med Res. 2023;12:e45903.
Kaul V, Enslin S, Gross SA. History of artificial intelligence in medicine. Gastrointest Endosc. 2020;92(4):807��812.
Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM, et al. Dermatologist��level classification of skin cancer with deep neural networks. Nature. 2017;542(7639):115��118.
Tschandl P, Rinner C, Apalla Z, Argenziano G, Codella N, Halpern A, et al. Human��computer collaboration for skin cancer recognition. Nat Med. 2020;26(8):1229��1234.
Haenssle HA, Fink C, Toberer F, Winkler J, Stolz W, Deinlein T, et al. Man against machine reloaded: performance of a market��approved convolutional neural network in classifying a broad spectrum of skin lesions in comparison with 96 dermatologists working under less artificial conditions. Ann Oncol. 2020;31(1):137��143.
Felmingham C, Pan Y, Kok Y, Kelly J, Gin D, Nguyen J, et al. Improving skin cancer management with ARTificial intelligence: a pre��post intervention trial of an artificial intelligence system used as a diagnostic aid for skin cancer management in a real��world specialist dermatology setting. J Am Acad Dermatol. 2023;88(5):1138��1142.
Smak Gregoor AM, Sangers TE, Bakker LJ, Hollestein L, Uyl��de Groot CA, Nijsten T, et al. An artificial intelligence based app for skin cancer detection evaluated in a population based setting. NPJ Digit Med. 2023;6(1):90.
Winkler JK, Blum A, Kommoss K, Enk A, Toberer F, Rosenberger A, et al. Assessment of diagnostic performance of dermatologists cooperating with a convolutional neural network in a prospective clinical study: human with machine. JAMA Dermatol. 2023;159(6):621��627.
Lee MS, Guo LN, Nambudiri VE. Towards gender equity in artificial intelligence and machine learning applications in dermatology. J Am Med Inform Assoc. 2022;29(2):400��403.
Niels K, Ternov ANC, Kampen PJT, Als G, Vestergaard T, Konge L, et al. Generalizability and usefulness of artificial intelligence for skin cancer diagnostics: an algorithm validation study. JEADV Clin Pract. 2022;1:344��354.
Winkler JK, Fink C, Toberer F, Enk A, Deinlein T, Hofmann��Wellenhof R, et al. Association between surgical skin markings in dermoscopic images and diagnostic performance of a deep learning convolutional neural network for melanoma recognition. JAMA Dermatol. 2019;155(10):1135��1141.
Felmingham CM, Adler NR, Ge Z, Morton RL, Janda M, Mar VJ. The importance of incorporating human factors in the design and implementation of artificial intelligence for skin cancer diagnosis in the real world. Am J Clin Dermatol. 2021;22(2):233��242.
Maron RC, Schlager JG, Haggenmuller S, von Kalle C, Utikal JS, Meier F, et al. A benchmark for neural network robustness in skin cancer classification. Eur J Cancer. 2021;155:191��199.
Food and Drug Administration UDoHaHS, editor. Guidance on medical device patient labelling; final guidance for industry and FDA reviewers. Rockville, USA: Food and Drug Administration UDoHaHS; 2001, editor.
Chen DK, Modi Y, Al��Aswad LA. Promoting transparency and standardization in ophthalmologic artificial intelligence: a call for artificial intelligence model card. Asia Pac J Ophthalmol. 2022;11(3):215��218.
Omoumi P, Ducarouge A, Tournier A, Harvey H, Kahn CE Jr, Louvet��de Verchere F, et al. To buy or not to buy��evaluating commercial AI solutions in radiology (the ECLAIR guidelines). Eur Radiol. 2021;31(6):3786��3796.
Hauser K, Kurz A, Haggenmuller S, Maron RC, von Kalle C, Utikal JS, et al. Explainable artificial intelligence in skin cancer recognition: a systematic review. Eur J Cancer. 2022;167:54��69.
Ghassemi M, Oakden��Rayner L, Beam AL. The false hope of current approaches to explainable artificial intelligence in health care. Lancet Digit Health. 2021;3(11):e745��e750.
Da Silva MFC, Goldenberg A, Singh D. Regulating the safety of health��related artificial intelligence. Health Policy. 2022;17(4):63��77.

APP2006551/NHMRC Centre for Research Excellence
/National Health and Medical Research Council
/Cancerfonden
2009923/NHMRC Synergy

Artificial Intelligence

Humans

Dermatology

Consensus

Product Labeling

Software

Delphi Technique

Australia

Journal Article

The EU medical device regulation: Implications for artificial intelligence-based medical device software in medical physics.Advancements in Clinical Evaluation and Regulatory Frameworks for AI-Driven Software as a Medical Device (SaMD).Changes in software as a medical device based on artificial intelligence technologies.Artificial intelligence in medicine - dermatology compared to other medical specialties in FDA-cleared software as medical device.Software as a Medical Device (SaMD) in Digestive Healthcare: Regulatory Challenges and Ethical Implications.CORE-MD clinical risk score for regulatory evaluation of artificial intelligence-based medical device software.The need for a system view to regulate artificial intelligence/machine learning-based software as medical device.Methodology for Conducting Post-Marketing Surveillance of Software as a Medical Device Based on Artificial Intelligence Technologies.[Medical device software requirements and evaluation].Medical Device Software: From Requirements to Certification.

No available data.