Information extraction on postoperative pulmonary embolism in neurosurgery: a research using artificial intelligence

Pulmonary embolism (PE) is one of the most dangerous but relatively rare and potentially preventable complications in neurosurgery. A reliable assessment of PE rate and risk factors in neurosurgery requires analyzing a significant number of cases accumulated over a long time.

PURPOSE OF THE STUDY: to evaluate the quality of information extraction about hospital PE after neurosurgical operations with a semi-automatic analysis of unstructured texts from the electronic health records of the FSAI «NMRC named after ac. N. N. Burdenko» of the ministry of health of Russia for a period of 18 years using artificial intelligence technologies.

MATERIALS AND METHODS: To accomplish our goal, we applied a set of technologies we developed using artificial intelligence (patent RU 2751993C 1). The textual medical records, initially typed in by doctors using the keyboard into the electronic health records system «E-med» of the FSAI «NMRC named after ac. N. N. Burdenko» of the ministry of health of Russia from 2000 to 2017 were analysed. Three independent experts (neurosurgeons) were involved to verify machine learning performance.

RESULTS. We analyzed 621 case histories that were most likely to contain information about PE. We observed moderate agreement (incomplete — in 32.4 % of cases) between 3 experts (Kappa Light coefficient = 0.568, p < 0.0001). Disagreements were resolved with the involvement of a third expert and the introduction of formal rules for texts labeling. The information extraction algorithm we proposed demonstrated good performance in detecting PE in electronic medical records (sensitivity = 0.996; specificity = 0.974; accuracy = 0.936; F1-score = 0.921).

CONCLUSION. minimization of the subjective factor using computer technologies improves the quality and reliability of retrospective studies over a huge set of medical records.

1. Khan NR, Patel PG, Sharpe JP, Lee SL, Sorenson J. Chemical venous thromboembolism prophylaxis in neurosurgical patients: an updated systematic review and meta-analysis. J Neurosurg. 2018;129(4):906–915. doi:10.3171/2017.2.jNS162040

2. Light RJ. Measures of response agreement for qualitative data: Some generalizations and alternatives. Psychol Bull. 1971;76(5):365–377. doi:10.1037/h0031643

3. Danilov G.V., Shifrin M.A., Potapov A.A., et al. Method for extracting information from unstructured texts written in natural language (patent rU2751993C1). published online july 21, 2021. accessed december 27, 2021. https://www.fips.ru/registers-doc-view/fips_servlet?dB=rUpaT&docNumber=2751993&TypeFile=html (In russ.).

4. danilov g, Kosyrkova a, Shults M, et al. Inter-rater reliability of Unstructured Text labeling: artificially vs. Naturally Intelligent approaches. Stud Health Technol Inform. 2021;281:118–122. doi:10.3233/SHTI210132

5. Danilov G, Ishankulov T, Kosyrkova A, et al. Semiautomatic Identification of pulmonary embolism in electronic Health records Through Sentence labeling. Stud Health Technol Inform. 2022;289:69–72. doi:10.3233/SHTI210861

6. Danilov G, Kosyrkova A, Shults M, et al. Inter-rater reliability of Unstructured Text labeling: artificially vs. Naturally Intelligent approaches. Stud Health Technol Inform. 2021;281:118–122. doi:10.3233/SHTI210132