Surveillance of Health Care-Associated Violence Using Natural Language Processing.

Mark Waltzman, Al Ozonoff, Kerri Ann Fournier, Jennifer Welcher, Carly Milliren, Assaf Landschaft, Jonathan Bulis, Amir A Kimia
Author Information
  1. Mark Waltzman: Boston Children's Hospital, Boston, Massachusetts.
  2. Al Ozonoff: Boston Children's Hospital, Boston, Massachusetts.
  3. Kerri Ann Fournier: Boston Children's Hospital, Boston, Massachusetts.
  4. Jennifer Welcher: Boston Children's Hospital, Boston, Massachusetts.
  5. Carly Milliren: Boston Children's Hospital, Boston, Massachusetts.
  6. Assaf Landschaft: Harvard Medical School, Boston, Massachusetts.
  7. Jonathan Bulis: Boston Children's Hospital, Boston, Massachusetts.
  8. Amir A Kimia: Boston Children's Hospital, Boston, Massachusetts.

Abstract


BACKGROUND AND OBJECTIVES: Patient and family violent outbursts toward staff, caregivers, or through self-harm, have increased during the ongoing behavioral health crisis. These health care-associated violence (HAV) episodes are likely under-reported. We sought to assess the feasibility of using nursing notes to identify under-reported HAV episodes.
METHODS: We extracted nursing notes across inpatient units at 2 hospitals for 2019: a pediatric tertiary care center and a community-based hospital. We used a workflow for narrative data processing using a natural language processing (NLP) assisted manual review process performed by domain experts (a nurse and a physician). We trained the NLP models on the tertiary care center data and validated it on the community hospital data. Finally, we applied these surveillance methods to real-time data for 2022 to assess reporting completeness of new cases.
RESULTS: We used 70���981 notes from the tertiary care center for model building and internal validation and 19���332 notes from the community hospital for external validation. The final community hospital model sensitivity was 96.8% (95% CI 90.6% to 100%) and a specificity of 47.1% (39.6% to 54.6%) compared with manual review. We identified 31 HAV episodes in July to December 2022, of which 26 were reportable in accordance with the hospital internal criteria. Only 7 of 26 cases were reported by employees using the self-reporting system, all of which were identified by our surveillance process.
CONCLUSIONS: NLP-assisted review is a feasible method for surveillance of under-reported HAV episodes, with implementation and usability that can be achieved even at a low information technology-resourced hospital setting.

References

  1. Multidiscip Respir Med. 2015 Mar 26;10(1):12 [PMID: 26052439]
  2. Pediatrics. 2023 May 1;151(5): [PMID: 37013707]
  3. J Patient Saf. 2020 Sep;16(3):187-193 [PMID: 27820722]
  4. Int Wound J. 2016 Oct;13(5):809-20 [PMID: 25469585]
  5. J Ment Health Policy Econ. 2020 Jun 1;23(2):61-75 [PMID: 32621726]
  6. Front Psychol. 2022 May 30;13:896156 [PMID: 35712196]
  7. J Patient Saf. 2024 Mar 1;20(2):119-124 [PMID: 38147064]
  8. Work. 2010;35(2):191-200 [PMID: 20164614]
  9. Pediatr Emerg Care. 2009 Dec;25(12):819-22 [PMID: 19952970]
  10. Pediatrics. 2024 Jan 1;153(1): [PMID: 38093653]
  11. J Nurs Manag. 2022 Sep;30(6):1852-1860 [PMID: 35767371]
  12. Health Informatics J. 2022 Oct-Dec;28(4):14604582221132429 [PMID: 36330784]
  13. CJEM. 2022 Dec;24(8):867-875 [PMID: 36344901]
  14. J Clin Nurs. 2018 May;27(9-10):1941-1949 [PMID: 29495119]
  15. Acad Emerg Med. 2022 Dec;29(12):1512-1514 [PMID: 36006174]
  16. Surgery. 2017 Mar;161(3):782-790 [PMID: 27788924]
  17. Lancet. 2020 Jun 6;395(10239):1743-1745 [PMID: 32445692]
  18. Acta Biomed. 2018 Feb 21;89(4-S):28-36 [PMID: 29644987]
  19. JAMA. 1995 Jul 5;274(1):29-34 [PMID: 7791255]
  20. Eur J Emerg Med. 2022 Apr 1;29(2):89-90 [PMID: 35210372]
  21. Am Surg. 2018 Sep 1;84(9):1422-1428 [PMID: 30268169]
  22. Dtsch Arztebl Int. 2015 Mar 6;112(10):159-65 [PMID: 25837740]
  23. Infect Control Hosp Epidemiol. 2011 Feb;32(2):101-14 [PMID: 21460463]
  24. Am J Emerg Med. 2021 Aug;46:550-555 [PMID: 33279330]
  25. Am J Emerg Med. 2019 Jan;37(1):109-113 [PMID: 29754963]
  26. Public Health. 2021 Jul;196:186-197 [PMID: 34246105]
  27. Pediatr Emerg Care. 2015 Jul;31(7):536-41 [PMID: 26148107]
  28. N Engl J Med. 2010 Nov 25;363(22):2124-34 [PMID: 21105794]
  29. Occup Environ Med. 2019 Dec;76(12):927-937 [PMID: 31611310]
  30. Clin Med (Lond). 2013 Dec;13(6):543-6 [PMID: 24298096]
  31. Diagnosis (Berl). 2018 Jun 27;5(2):63-69 [PMID: 29858901]
  32. Acta Biomed. 2019 Dec 23;90(4):621-624 [PMID: 31910199]
  33. J Biomed Inform. 2009 Apr;42(2):390-405 [PMID: 19232399]
  34. Am J Emerg Med. 2021 Aug;46:495-498 [PMID: 33261949]
  35. N Engl J Med. 2016 Aug 18;375(7):e14 [PMID: 27532857]
  36. BMJ Qual Saf. 2014 Sep;23(9):727-31 [PMID: 24742777]
  37. Am J Emerg Med. 2021 Aug;46:34-37 [PMID: 33714052]
  38. Biomed Res Int. 2019 May 28;2019:5430870 [PMID: 31275976]
  39. J Nurs Scholarsh. 2022 Nov;54(6):762-771 [PMID: 35819267]
  40. J Nurs Scholarsh. 2020 May;52(3):281-291 [PMID: 32212311]
  41. BMJ Qual Saf. 2013 Oct;22(10):809-15 [PMID: 24048616]

Grants

  1. R01 HS026246/AHRQ HHS

MeSH Term

Humans
Feasibility Studies
Hospitals, Community
Hospitals, Pediatric
Natural Language Processing
Population Surveillance
Tertiary Care Centers
Workplace Violence

Word Cloud

Created with Highcharts 10.0.0hospitalHAVepisodesnotesdataunder-reportedusingtertiarycarecenterreviewcommunitysurveillance6%healthassessnursingusedprocessingNLPmanualprocess2022casesmodelinternalvalidationidentified26BACKGROUNDANDOBJECTIVES:Patientfamilyviolentoutburststowardstaffcaregiversself-harmincreasedongoingbehavioralcrisiscare-associatedviolencelikelysoughtfeasibilityidentifyMETHODS:extractedacrossinpatientunits2hospitals2019:pediatriccommunity-basedworkflownarrativenaturallanguageassistedperformeddomainexpertsnursephysiciantrainedmodelsvalidatedFinallyappliedmethodsreal-timereportingcompletenessnewRESULTS:70���981building19���332externalfinalsensitivity968%95%CI90100%specificity471%3954compared31JulyDecemberreportableaccordancecriteria7reportedemployeesself-reportingsystemCONCLUSIONS:NLP-assistedfeasiblemethodimplementationusabilitycanachievedevenlowinformationtechnology-resourcedsettingSurveillanceHealthCare-AssociatedViolenceUsingNaturalLanguageProcessing

Similar Articles

Cited By

No available data.