Explainable deep-learning framework: decoding brain states and prediction of individual performance in false-belief task at early childhood stage.

Km Bhavna, Azman Akhter, Romi Banerjee, Dipanjan Roy
Author Information
  1. Km Bhavna: Department of Computer Science and Engineering, IIT Jodhpur, Karwar, Rajasthan, India.
  2. Azman Akhter: Cognitive Brain Dynamics Lab, National Brain Research Centre, Manesar, Gurugram, India.
  3. Romi Banerjee: Department of Computer Science and Engineering, IIT Jodhpur, Karwar, Rajasthan, India.
  4. Dipanjan Roy: Cognitive Brain Dynamics Lab, National Brain Research Centre, Manesar, Gurugram, India.

Abstract

Decoding of cognitive states aims to identify individuals' brain states and brain fingerprints to predict behavior. Deep learning provides an important platform for analyzing brain signals at different developmental stages to understand brain dynamics. Due to their internal architecture and feature extraction techniques, existing machine-learning and deep-learning approaches are suffering from low classification performance and explainability issues that must be improved. In the current study, we hypothesized that even at the early childhood stage (as early as 3-years), connectivity between brain regions could decode brain states and predict behavioral performance in false-belief tasks. To this end, we proposed an explainable deep learning framework to decode brain states (Theory of Mind and Pain states) and predict individual performance on ToM-related false-belief tasks in a developmental dataset. We proposed an explainable spatiotemporal connectivity-based Graph Convolutional Neural Network (Ex-stGCNN) model for decoding brain states. Here, we consider a developmental dataset, = 155 (122 children; 3-12 yrs and 33 adults; 18-39 yrs), in which participants watched a short, soundless animated movie, shown to activate Theory-of-Mind (ToM) and pain networs. After scanning, the participants underwent a ToM-related false-belief task, leading to categorization into the pass, fail, and inconsistent groups based on performance. We trained our proposed model using Functional Connectivity (FC) and Inter-Subject Functional Correlations (ISFC) matrices separately. We observed that the stimulus-driven feature set (ISFC) could capture ToM and Pain brain states more accurately with an average accuracy of 94%, whereas it achieved 85% accuracy using FC matrices. We also validated our results using five-fold cross-validation and achieved an average accuracy of 92%. Besides this study, we applied the SHapley Additive exPlanations (SHAP) approach to identify brain fingerprints that contributed the most to predictions. We hypothesized that ToM network brain connectivity could predict individual performance on false-belief tasks. We proposed an Explainable Convolutional Variational Auto-Encoder (Ex-Convolutional VAE) model to predict individual performance on false-belief tasks and trained the model using FC and ISFC matrices separately. ISFC matrices again outperformed the FC matrices in prediction of individual performance. We achieved 93.5% accuracy with an F1-score of 0.94 using ISFC matrices and achieved 90% accuracy with an F1-score of 0.91 using FC matrices.

Keywords

References

  1. Neural Comput. 1997 Nov 15;9(8):1735-80 [PMID: 9377276]
  2. IEEE Trans Pattern Anal Mach Intell. 2023 May;45(5):5782-5799 [PMID: 36063508]
  3. Nature. 2006 Jul 13;442(7099):195-8 [PMID: 16838020]
  4. PLoS Comput Biol. 2018 Nov 29;14(11):e1006565 [PMID: 30496171]
  5. Hum Brain Mapp. 2022 Mar;43(4):1403-1418 [PMID: 34859934]
  6. Dev Psychol. 2007 Sep;43(5):1124-39 [PMID: 17723040]
  7. BMJ. 2005 Aug 20;331(7514):433-4 [PMID: 15937056]
  8. Front Neurosci. 2019 Oct 30;13:1165 [PMID: 31736698]
  9. IEEE Trans Neural Netw Learn Syst. 2024 Jun;35(6):7312-7323 [PMID: 36099220]
  10. Nat Commun. 2016 Jul 18;7:12141 [PMID: 27424918]
  11. Cogn Sci. 2011 Sep-Oct;35(7):1282-304 [PMID: 21884221]
  12. Neuroimage. 2021 May 1;231:117847 [PMID: 33582272]
  13. Soc Cogn Affect Neurosci. 2014 Jun;9(6):817-24 [PMID: 23552077]
  14. Brain Sci. 2022 Aug 17;12(8): [PMID: 36009157]
  15. PLoS Comput Biol. 2016 Jun 16;12(6):e1004994 [PMID: 27310288]
  16. Neuroimage. 2019 Aug 1;196:126-141 [PMID: 30974241]
  17. Sci Rep. 2019 Oct 3;9(1):14286 [PMID: 31582792]
  18. Brain Struct Funct. 2015 Mar;220(2):587-604 [PMID: 24915964]
  19. Neuroimage. 2021 Jul 15;235:117963 [PMID: 33813007]
  20. Trends Cogn Sci. 2006 Feb;10(2):59-63 [PMID: 16406760]
  21. Nat Commun. 2018 Mar 12;9(1):1027 [PMID: 29531321]
  22. Med Image Anal. 2021 Dec;74:102233 [PMID: 34655865]
  23. Molecules. 2021 Feb 19;26(4): [PMID: 33669834]
  24. Neuroimage. 2016 Feb 1;126:39-48 [PMID: 26589334]
  25. Neuroimage. 2019 Nov 15;202:116059 [PMID: 31362049]
  26. Neuroimage. 1995 Jun;2(2):89-101 [PMID: 9343592]
  27. Neuroimage. 2019 Jan 1;184:335-348 [PMID: 30237036]
  28. Hum Brain Mapp. 2023 May;44(7):2921-2935 [PMID: 36852610]
  29. Adv Neural Inf Process Syst. 2019 Dec;32:9240-9251 [PMID: 32265580]
  30. Neuron. 2011 Dec 8;72(5):692-7 [PMID: 22153367]
  31. Soc Cogn Affect Neurosci. 2019 Aug 7;14(6):667-685 [PMID: 31099394]
  32. Med Image Comput Comput Assist Interv. 2018 Sep;11072:320-328 [PMID: 30320311]
  33. Front Neuroinform. 2017 Oct 17;11:61 [PMID: 29089883]
  34. Hum Brain Mapp. 2018 Dec;39(12):4939-4948 [PMID: 30144210]
  35. Science. 2001 Sep 28;293(5539):2425-30 [PMID: 11577229]
  36. BMC Res Notes. 2017 Sep 6;10(1):446 [PMID: 28877742]
  37. Cereb Cortex. 2018 Sep 1;28(9):3065-3081 [PMID: 28981593]
  38. PLoS Biol. 2006 May;4(5):e125 [PMID: 16594732]
  39. Neuroimage. 2002 Sep;17(1):184-200 [PMID: 12482076]
  40. Front Neurosci. 2022 Sep 01;16:875828 [PMID: 36117636]
  41. Philos Trans R Soc Lond B Biol Sci. 2001 Aug 29;356(1412):1293-322 [PMID: 11545704]
  42. Nat Neurosci. 2017 Jan;20(1):107-114 [PMID: 27798630]
  43. IEEE Trans Pattern Anal Mach Intell. 2019 Aug;41(8):2008-2026 [PMID: 30596568]
  44. KDD. 2016 Aug;2016:855-864 [PMID: 27853626]
  45. Neuroimage. 2020 Aug 1;216:116461 [PMID: 31843711]
  46. PLoS One. 2018 Sep 13;13(9):e0204056 [PMID: 30212588]
  47. Proc Natl Acad Sci U S A. 2016 May 3;113(18):E2474-5 [PMID: 27095849]
  48. Cognition. 1997 Mar;62(3):291-324 [PMID: 9187061]
  49. Proc Natl Acad Sci U S A. 2016 May 3;113(18):E2476-9 [PMID: 27095848]
  50. Psychol Sci. 2009 Nov;20(11):1364-72 [PMID: 19883493]
  51. IEEE J Biomed Health Inform. 2024 Mar;28(3):1494-1503 [PMID: 38157464]
  52. Proc Natl Acad Sci U S A. 2015 Dec 8;112(49):15250-5 [PMID: 26582792]
  53. Hum Brain Mapp. 2020 Apr 15;41(6):1505-1519 [PMID: 31816152]
  54. Science. 2004 Mar 12;303(5664):1634-40 [PMID: 15016991]
  55. Neuropsychologia. 2017 Oct;105:70-83 [PMID: 28057458]
  56. Neurosci Biobehav Rev. 2018 Sep;92:318-337 [PMID: 29944961]

Word Cloud

Created with Highcharts 10.0.0brainstatesperformancefalse-beliefmatricesusingpredictindividualFCISFCaccuracytasksproposedmodelachieveddevelopmentalearlydecodingToMtaskidentifyfingerprintslearningfeaturedeep-learningstudyhypothesizedchildhoodstageconnectivitydecodeexplainablePainToM-relateddatasetConvolutionalyrsparticipantspaintrainedFunctionalseparatelyaverageExplainablepredictionF1-score0networksDecodingcognitiveaimsindividuals'behaviorDeepprovidesimportantplatformanalyzingsignalsdifferentstagesunderstanddynamicsDueinternalarchitectureextractiontechniquesexistingmachine-learningapproachessufferinglowclassificationexplainabilityissuesmustimprovedcurrenteven3-yearsregionsbehavioralenddeepframeworkTheoryMindspatiotemporalconnectivity-basedGraphNeuralNetworkEx-stGCNNconsider=155122children3-1233adults18-39watchedshortsoundlessanimatedmovieshownactivateTheory-of-MindnetworsscanningunderwentleadingcategorizationpassfailinconsistentgroupsbasedConnectivityInter-SubjectCorrelationsobservedstimulus-drivensetcaptureaccurately94%whereas85%alsovalidatedresultsfive-foldcross-validation92%BesidesappliedSHapleyAdditiveexPlanationsSHAPapproachcontributedpredictionsnetworkVariationalAuto-EncoderEx-ConvolutionalVAEoutperformed935%9490%91framework:graphneuraltheorymind

Similar Articles

Cited By