Using Clinical Natural Language Processing for Health Outcomes Research: Overview and Actionable Suggestions for Future Advances

Sumithra Velupillai

[0]

[0]

[0]

[0]

[0]

[0]

[0]

[0]

[0]

[0]

[0]

[0]

Journal of Biomedical Informatics, pp. 11-19, 2018.

Cited by: 27|Bibtex|Views31|DOI:https://doi.org/10.1016/j.jbi.2018.10.005

WOS EI

Other Links: pubmed.ncbi.nlm.nih.gov|dblp.uni-trier.de|academic.microsoft.com|www.sciencedirect.com

Keywords:

Natural Language ProcessingInformation extractionText analyticsEvaluationClinical informaticsMore(3+)

Weibo:

For clinical Natural Language Processing method development to become more integral in clinical outcomes research, there is a need to develop evaluation workbenches that can be used by clinicians to better understand the underlying parts of an Natural Language Processing system a...

Abstract:

•Natural Language Processing methods can be impactful in health outcomes research.•More rigorous evaluation practices are necessary to advance this field.•Synthetic data and novel governance structures could address data access challenges.•Evaluation workbenches that allow for extrinsic and detailed evaluation are needed.•Structured proto...More

Code:

Data:

Insight

Full Text

PPT (Upload PPT)

Similar

Reference

Introduction

Appropriate utilization of large data sources such as Electronic

Health Record databases could have a dramatic impact on health care research and delivery.
The above include recommendations to address the key challenges of limited collaboration, lack of shared resources and evaluation-approaches of crucial tasks, such as de-identification, recognition and classification of medical concepts, semantic modifiers, and temporal information.
These challenges have been addressed by the organization of several shared tasks
These include the Informatics for Integrating Biology and the Bedside (i2b2) challenges [5–9], the Conference and Labs of the Evaluation Forum (CLEF) eHealth challenges [10–13], and the Semantic Evaluation (SemEval) challenges [14–16].
These efforts have enabled a valuable platform for international NLP method development

Highlights

Appropriate utilization of large data sources such as Electronic

Health Record databases could have a dramatic impact on health care research and delivery
Studies of diagnostic tools are most similar to Natural Language Processing method development - testing whether a history item, examination finding or test result is associated with a subsequent diagnosis
For clinical Natural Language Processing method development to become more integral in clinical outcomes research, there is a need to develop evaluation workbenches that can be used by clinicians to better understand the underlying parts of an Natural Language Processing system and its impact on outcomes
Work in the general Natural Language Processing domain could be inspirational for such development, for instance integrating methods to analyse the effect of Natural Language Processing pipeline steps in downstream tasks such as the effect of dependency parsing approaches [84]
Facilitating the integration of domain knowledge in Natural Language Processing system development can be done by providing support for formalized knowledge representations that can be used in subsequent Natural Language Processing method development [86]
We propose a minimal structured protocol that could be used when reporting clinical Natural Language Processing method development and its evaluation, to enable transparency and reproducibility

Methods

Methods for developing shareable data

Risks for compromised privacy are evident in analyzing text from health records and in big data health research more generally.
Methods for developing shareable data.
Risks for compromised privacy are evident in analyzing text from health records and in big data health research more generally.
Synthetic data has been successful in tasks such as dialogue generation [59] and is a promising direction at least as a complement for method development where access to data is challenging

Results

Evaluation paradigms

All empirical research studies need to be evaluated in order to allow for scientific assessment of a study.
The most basic underlying construction for quantitative validation in both fields is a 2 × 2 contingency table, where the number of correctly and incorrectly assigned values for a given binary outcome or classification label is compared with a gold standard, i.e. the set of ’true’ or correct values
This table can be used to calculate performance metrics such as precision (Positive Predictive Value), recall, accuracy, F-score, and specificity.
Facilitating the integration of domain knowledge in NLP system development can be done by providing support for formalized knowledge representations that can be used in subsequent NLP method development [86]

Conclusion

The authors have sought to provide a broad outline of the current state-ofthe-art, opportunities, challenges, and needs in the use of NLP for health outcomes research, with a particular focus on evaluation methods.
The authors have outlined methodological aspects from a clinical as well as an NLP perspective and identify three main challenges: data availability, evaluation workbenches and reporting standards.
The authors propose a minimal structured protocol that could be used when reporting clinical NLP method development and its evaluation, to enable transparency and reproducibility.
The authors envision further advances in methods for data access, evaluation methods that move beyond current intrinsic metrics and move closer to clinical practice and utility, and in transparent and reproducible method development

Summary

Introduction:
Appropriate utilization of large data sources such as Electronic

Health Record databases could have a dramatic impact on health care research and delivery.
The above include recommendations to address the key challenges of limited collaboration, lack of shared resources and evaluation-approaches of crucial tasks, such as de-identification, recognition and classification of medical concepts, semantic modifiers, and temporal information.
These challenges have been addressed by the organization of several shared tasks
These include the Informatics for Integrating Biology and the Bedside (i2b2) challenges [5–9], the Conference and Labs of the Evaluation Forum (CLEF) eHealth challenges [10–13], and the Semantic Evaluation (SemEval) challenges [14–16].
These efforts have enabled a valuable platform for international NLP method development
Methods:
Methods for developing shareable data

Risks for compromised privacy are evident in analyzing text from health records and in big data health research more generally.
Methods for developing shareable data.
Risks for compromised privacy are evident in analyzing text from health records and in big data health research more generally.
Synthetic data has been successful in tasks such as dialogue generation [59] and is a promising direction at least as a complement for method development where access to data is challenging
Results:
Evaluation paradigms

All empirical research studies need to be evaluated in order to allow for scientific assessment of a study.
The most basic underlying construction for quantitative validation in both fields is a 2 × 2 contingency table, where the number of correctly and incorrectly assigned values for a given binary outcome or classification label is compared with a gold standard, i.e. the set of ’true’ or correct values
This table can be used to calculate performance metrics such as precision (Positive Predictive Value), recall, accuracy, F-score, and specificity.
Facilitating the integration of domain knowledge in NLP system development can be done by providing support for formalized knowledge representations that can be used in subsequent NLP method development [86]
Conclusion:
The authors have sought to provide a broad outline of the current state-ofthe-art, opportunities, challenges, and needs in the use of NLP for health outcomes research, with a particular focus on evaluation methods.
The authors have outlined methodological aspects from a clinical as well as an NLP perspective and identify three main challenges: data availability, evaluation workbenches and reporting standards.
The authors propose a minimal structured protocol that could be used when reporting clinical NLP method development and its evaluation, to enable transparency and reproducibility.
The authors envision further advances in methods for data access, evaluation methods that move beyond current intrinsic metrics and move closer to clinical practice and utility, and in transparent and reproducible method development

Funding

This work is the result of an international workshop held at the Institute of Psychiatry, Psychology and Neuroscience, King’s College London, on April 28th 2017, with financial support from the European Science Foundation (ESF) Research Networking Programme Evaluating Information Access Systems: http://eliasnetwork.eu/
SV is supported by the Swedish Research Council (2015-00359) and the Marie Skłodowska Curie Actions, Cofund, Project INCA 600398
RD is funded by a Clinician Scientist Fellowship (research project e-HOST-IT) from the Health Foundation in partnership with the Academy of Medical Sciences
JD is supported by a Medical Research Council (MRC) Clinical Research Training Fellowship (MR/L017105/1)
KM is funded by the Wellcome Trust Seed Award in Science [109823/Z/15/Z]
RS and AR is part-funded by the National Institute for Health Research (NIHR) Biomedical Research Centre at South London and Maudsley NHS Foundation Trust and King’s College London
David (PJ) Osborn, Joseph (F) Hayes and Anoop (D) Shah are supported by the National Institute for Health Research University College London Hospitals Biomedical Research Centre
Prof Osborn is also in part supported by the NIHR Collaboration for Leadership in Applied Health Research and Care (CLAHRC) North Thames at Bart’s Health NHS Trust

Reference

A. Névéol, P. Zweigenbaum, Clinical Natural Language Processing in 2014: foundational methods supporting efficient healthcare, Yearb. Med. Inform. 10 (1) (2015) 194–198.
S. Velupillai, D. Mowery, B.R. South, M. Kvist, H. Dalianis, Recent advances in clinical natural language processing in support of semantic analysis, IMIA Yearb. Med. Inform. 10 (2015) 183–193.
W.W. Chapman, P.M. Nadkarni, L. Hirschman, L.W. D’Avolio, G.K. Savova, O. Uzuner, Overcoming barriers to NLP for clinical text: the role of shared tasks and the need for additional creative solutions, J. Am. Med. Inform. Assoc. 18 (5) (2011) 540–543.
C. Friedman, T.C. Rindflesch, M. Corn, Natural language processing: State of the art and prospects for significant progress, a workshop sponsored by the National Library of Medicine, J. Biomed. Inform. 46 (5) (2013) 765–773, https://doi.org/10.1016/j.jbi.2013.06.004.
Ö. Uzuner, Y. Luo, P. Szolovits, Evaluating the state-of-the-art in automatic deidentification, J. Am. Med. Inform. Assoc. 14 (5) (2007) 550, https://doi.org/10.1197/jamia.M2444.
Ö. Uzuner, I. Solti, E. Cadag, Extracting medication information from clinical text, J. Am. Med. Inform. Assoc. 17 (5) (2010) 514, https://doi.org/10.1136/jamia.2010.003947.
Ö. Uzuner, B.R. South, S. Shen, S.L. DuVall, 2010 i2b2/VA challenge on concepts, assertions, and relations in clinical text, J. Am. Med. Inform. Assoc. 18 (5) (2011) 552, https://doi.org/10.1136/amiajnl-2011-000203.
Ö. Uzuner, A. Bodnari, S. Shen, T. Forbush, J. Pestian, B.R. South, Evaluating the state of the art in coreference resolution for electronic medical records, J. Am. Med. Inform. Assoc. 19 (5) (2012) 786, https://doi.org/10.1136/amiajnl-2011-000784.
W. Sun, A. Rumshisky, O. Uzuner, Evaluating temporal relations in clinical text: 2012 i2b2 Challenge, J. Am. Med. Inform. Assoc. 20 (5) (2013) 806, https://doi.org/10.1136/amiajnl-2013-001628.
H. Suominen, S. Salanterä, S. Velupillai, W. Chapman, G. Savova, N. Elhadad, S. Pradhan, B. South, D. Mowery, G. Jones, J. Leveling, L. Kelly, L. Goeuriot, D. Martinez, G. Zuccon, Overview of the ShARe/CLEF eHealth evaluation lab 2013, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 8138 LNCS (2013) 212–231. https://doi.org/10.1007/978-3-642-40802-1_24.
L. Kelly, L. Goeuriot, H. Suominen, T. Schreck, G. Leroy, D. Mowery, S. Velupillai, W. Chapman, D. Martinez, G. Zuccon, J. Palotti, Overview of the ShARe/CLEF eHealth evaluation lab 2014, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 8685 LNCS (2014) 172–191. https://doi.org/10.1007/978-3-319-11382-1_17.
L. Goeuriot, L. Kelly, H. Suominen, L. Hanlen, A. Névéol, C. Grouin, J. Palotti, G. Zuccon, Overview of the CLEF eHealth Evaluation Lab 2015, Springer International Publishing, 2015 pp. 429–443.
L. Kelly, L. Goeuriot, H. Suominen, A. Névéol, J. Palotti, G. Zuccon, Overview of the CLEF eHealth Evaluation Lab 2016, Springer International Publishing, 2016, https://doi.org/10.1007/978-3-319-44564-9_24 pp.255–266.
N. Elhadad, S. Pradhan, S. Gorman, S. Manandhar, W. Chapman, G. Savova, SemEval-2015 task 14: Analysis of clinical text, Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), Association for Computational Linguistics, Denver, Colorado, 2015, pp. 303–310.
S. Bethard, L. Derczynski, G. Savova, J. Pustejovsky, M. Verhagen, SemEval-2015 task 6: Clinical TempEval, Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), Association for Computational Linguistics, Denver, Colorado, 2015, pp. 806–814.
S. Bethard, G. Savova, W.-T. Chen, L. Derczynski, J. Pustejovsky, M. Verhagen, Semeval-2016 task 12: Clinical tempeval, Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), Association for Computational Linguistics, San Diego, California, 2016, pp. 1052–1062.
H. Wu, G. Toti, K.I. Morley, Z.M. Ibrahim, A. Folarin, R. Jackson, I. Kartoglu, A. Agrawal, C. Stringer, D. Gale, G. Gorrell, A. Roberts, M. Broadbent, R. Stewart, R.J. Dobson, SemEHR: a general-purpose semantic search system to surface semantic data from clinical notes for tailored care, trial recruitment, and clinical research, J. Am. Med. Inform. Assoc. 25 (5) (2018) 530–537, https://doi.org/10.1093/jamia/ocx160 arXiv:/oup/backfile/content_public/journal/jamia/25/5/10.1093_jamia_ocx160/3/ocx160.pdf.
M. De Choudhury, S. De, Mental health discourse on reddit: self-disclosure, social support, and anonymity, in: Eighth International AAAI Conference on Weblogs and Social Media, 2014.
U. Pavalanathan, M. De Choudhury, Identity Management and Mental Health Discourse in Social Media, in: Proceedings of the International World-Wide Web Conference. International WWW Conference 2015 (Companion), 2015, pp. 315–321.
D. Mowery, H. Smith, T. Cheney, G. Stoddard, G. Coppersmith, C. Bryan, M. Conway, Understanding depressive symptoms and psychosocial stressors on twitter: a corpus-based study, J. Med. Internet Res. 19 (2) (2017) e48, https://doi.org/10.2196/jmir.6895.
G. Gkotsis, A. Oellrich, S. Velupillai, M. Liakata, T.J.P. Hubbard, R.J.B. Dobson, R. Dutta, Characterisation of mental health conditions in social media using Informed Deep Learning, Sci. Rep. 7 (2017) 45141.
C. Howes, M. Purver, R. McCabe, Linguistic Indicators of Severity and Progress in Online Text-based Therapy for Depression, Proceedings of the Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality, Association for Computational Linguistics, Baltimore, Maryland, USA, 2014, pp. 7–16.
D. Angus, B. Watson, A. Smith, C. Gallois, J. Wiles, Visualising conversation structure across time: insights into effective doctor-patient consultations, PLOS One 7 (6) (2012) 1–12, https://doi.org/10.1371/journal.pone.0038014.
T. Althoff, K. Clark, J. Leskovec, Natural Language Processing for Mental Health: Large Scale Discourse Analysis of Counseling Conversations, CoRR abs/1605. 04462. URL <http://arxiv.org/abs/1605.04462>.
E. Yelland, What text mining analysis of psychotherapy records can tell us about therapy process and outcome, Ph.D. thesis, UCL (University College London), 2017.
J.P. Pestian, P. Matykiewicz, M. Linn-Gust, B. South, O. Uzuner, J. Wiebe, K.B. Cohen, J. Hurdle, C. Brew, Sentiment analysis of suicide notes: a shared task, Biomedical Informatics Insights 5 (2012) 3–16, https://doi.org/10.4137/BII.S9042.
D.N. Milne, G. Pink, B. Hachey, R.A. Calvo, CLPsych 2016 shared task: triaging content in online peer-support forums, Proceedings of the Third Workshop on Computational Linguistics and Clinical Psychology, Association for Computational Linguistics, San Diego, CA, USA, 2016, pp. 118–127.
M. Filannino, A. Stubbs, Ö. Uzuner, Symptom severity prediction from neuropsychiatric clinical records: Overview of 2016 {CEGS} N-GRID shared tasks Track 2, J. Biomed. Inform. (2017), https://doi.org/10.1016/j.jbi.2017.04.017.
H. Suominen, S. Pyysalo, M. Hiissa, F. Ginter, S. Liu, D. Marghescu, T. Pahikkala, B. Back, H. Karsten, T. Salakoski, Performance evaluation measures for text mining, in: M. Song, Y.-F. Wu (Eds.), Handbook of Research on Text and Web Mining Technologies, vol. II, IGI Global, Hershey, Pennsylvania, USA, 2008, pp. 724–747 (Chapter XLI).
E. Steyerberg, Clinical Prediction Models, Springer-Verlag, New York, 2009.
G. Collins, J. Reitsma, D. Altman, K. Moons, Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (tripod): The tripod statement, Ann. Intern. Med. 162 (1) (2015) 55–63, https://doi.org/10.7326/M140697 arXiv:/data/journals/aim/931895/0000605-201501060-00009.pdf, URL +https://doi.org/10.7326/M14-0697.
K. Sparck Jones, Evaluating Natural Language Processing Systems An Analysis and Review, Lecture Notes in Computer Science, Lecture Notes in Artificial Intelligence, 1083, 1995.
P. Paroubek, S. Chaudiron, L. Hirschman, Editorial: Principles of Evaluation in Natural Language Processing, TAL 48 (1) (2007) 7–31 <http://www.atala.org/ Editorial>.
L. Dybkjaer, Evaluation of Text and Speech Systems, Text, Speech and Language Technology, 37, 2007.
P.R. Cohen, A.E. Howe, Toward AI research methodology: three case studies in evaluation, IEEE Trans. Syst., Man Cybern. 19 (3) (1989) 634–646.
G. Hripcsak, A.S. Rothschild, Agreement, the f-measure, and reliability in information retrieval, J. Am. Med. Inform. Assoc. 12 (3) (2005) 296–298.
B.A. Goldstein, A.M. Navar, M.J. Pencina, J.P.A. Ioannidis, Opportunities and challenges in developing risk prediction models with electronic health records data: a systematic review, J. Am. Med. Inform Assoc.: JAMIA 24 (1) (2017) 198–208, https://doi.org/10.1093/jamia/ocw042.
A. Roberts, Language, structure, and reuse in the electronic health record, AMA J. Ethics 19 (3) (2017) 281–288.
S.T. Rosenbloom, J.C. Denny, H. Xu, N. Lorenzi, W.W. Stead, K.B. Johnson, Data from clinical notes: a perspective on the tension between structure and flexible documentation, J. Am. Med. Inform. Assoc. 18 (2) (2011) 181–186.
T. Greenhalgh, H.W.W. Potts, G. Wong, P. Bark, D. Swinglehurst, Tensions and paradoxes in electronic patient record research: a systematic literature review using the meta-narrative method, Milbank Q 87 (4) (2009) 729–788.
G. Carter, A. Milner, K. McGill, J. Pirkis, N. Kapur, M.J. Spittal, Predicting suicidal behaviours using clinical instruments: systematic review and meta-analysis of positive predictive values for risk scales, Br. J. Psychiatry 210 (6) (2017) 387–395, https://doi.org/10.1192/bjp.bp.116.182717 <http://bjp.rcpsych.org/content/210/6/387>.
J. Downs, S. Velupillai, G. Gkotsis, R. Holden, M. Kikoler, H. Dean, A. Fernandes, R. Dutta, Detection of suicidality in adolescents with autism spectrum disorders: developing a natural language processing approach for use in electronic health records, AMIA annual Symposium, AMIA, 2017, pp. 641–649.
S. Gange, E.T. Golub, From smallpox to big data: the next 100 years of epidemiologic methods, Am. J. Epidemiol. 183 (5) (2016) 423–426, https://doi.org/10.1093/aje/kwv150.
S.M. Lynch, J.H. Moore, A call for biological data mining approaches in epidemiology, BioData Mining 9 (1) (2016) 1, https://doi.org/10.1186/s13040-0150079-8 <http://www.biodatamining.org/content/9/1/1>.
J. Bell, C. Kilic, R. Prabakaran, Y.Y. Wang, R. Wilson, M. Broadbent, A. Kumar, V. Curtis, Use of electronic health records in identifying drug and alcohol misuse among psychiatric in-patients, The Psychiatrist 37 (1) (2013) 15–20, https://doi.org/10.1192/pb.bp.111.038240.
E. Ford, J.A. Carroll, H.E. Smith, D. Scott, J.A. Cassell, Extracting information from the text of electronic medical records to improve case detection: a systematic review, J. Am. Med. Inform. Assoc. 23 (5) (2016) 1007–1015, https://doi.org/10.1093/jamia/ocv180.
K.P. Liao, T. Cai, G.K. Savova, S.N. Murphy, E.W. Karlson, A.N. Ananthakrishnan, V.S. Gainer, S.Y. Shaw, Z. Xia, P. Szolovits, S. Churchill, I. Kohane, Development of phenotype algorithms using electronic medical records and incorporating natural language processing, BMJ 350 (2015) h1885, https://doi.org/10.1136/bmj.h1885.
G. Hripcsak, D.J. Albers, Next-generation phenotyping of electronic health records, J. Am. Med. Inform. Assoc. 20 (1) (2013) 117–121, https://doi.org/10.1136/amiajnl-2012-001145.
K.M. Newton, P.L. Peissig, A.N. Kho, S.J. Bielinski, R.L. Berg, V. Choudhary, M. Basford, C.G. Chute, I.J. Kullo, R. Li, J.a. Pacheco, L.V. Rasmussen, L. Spangler, J.C. Denny, Validation of electronic medical record-based phenotyping algorithms: results and lessons learned from the eMERGE network, J. Am. Med. Inform. Assoc. 20 (e1) (2013) e147–54, https://doi.org/10.1136/amiajnl-2012-000896.
K.I. Morley, J. Wallace, S.C. Denaxas, R.J. Hunter, R.S. Patel, P. Perel, A.D. Shah, A.D. Timmis, R.J. Schilling, H. Hemingway, Defining disease phenotypes using national linked electronic health records: a case study of atrial fibrillation, PLoS One 9 (11) (2014) e110900, https://doi.org/10.1371/journal.pone.0110900.
G. Peat, R.D. Riley, P. Croft, K.I. Morley, P.a. Kyzas, K.G.M. Moons, P. Perel, E.W. Steyerberg, S. Schroter, D.G. Altman, H. Hemingway, Improving the transparency of prognosis research: the role of reporting, data sharing, registration, and protocols, PLoS Med. 11 (7) (2014) e1001671, https://doi.org/10.1371/journal.pmed.1001671.
S. Wu, T. Miller, J. Masanz, M. Coarr, S. Halgrim, D. Carrell, C. Clark, Negation’s not solved: generalizability versus optimizability in clinical natural language processing, PLOS One 9 (11) (2014) 1–11, https://doi.org/10.1371/journal.pone.0112774.
D. Demner-Fushman, W.W. Chapman, C.J. McDonald, What can natural language processing do for clinical decision support? J. Biomed. Inform. 42 (5) (2009) 760–772, https://doi.org/10.1016/j.jbi.2009.08.007.
K. Zheng, V.G.V. Vydiswaran, Y. Liu, Y. Wang, A. Stubbs, O. Uzuner, A.E. Gururaj, S. Bayer, J. Aberdeen, A. Rumshisky, S. Pakhomov, H. Liu, H. Xu, Ease of adoption of clinical natural language processing software: an evaluation of five systems, J. Biomed. Inform. 58 Suppl (2015) S189–96.
D.R. Kaufman, B. Sheehan, P. Stetson, A.R. Bhatt, A.I. Field, C. Patel, J.M. Maisel, Natural language processing-enabled and conventional data capture methods for input to electronic health records: a comparative usability study, JMIR Med. Inform. 4 (4) (2016) e35.
H. Suominen, H. Müller, L. Ohno-Machado, S. Salanterä, G. Schreier, L. Hanlen, Prerequisites for International Exchanges of Health Information: Comparison of Australian, Austrian, Finnish, Swiss, and US Privacy Policies, in: TBA (Ed.), Medinfo
H. Suominen, L. Zhou, L. Hanlen, G. Ferraro, Benchmarking clinical speech recognition and information extraction: new data, methods, and evaluations, JMIR Med. Inform. 3 (2) (2015) e19, https://doi.org/10.2196/medinform.4321.
E. Aramaki, M. Morita, Y. Kano, T. Ohkuma, Overview of the NTCIR-11 MedNLP task, in: Proceedings of the 11th NTCIR Conference, NII Testbeds and Community for Information access Research (NTCIR), Tokyo, Japan, 2014, pp. 147–154.
I. Serban, A. Sordoni, R. Lowe, L. Charlin, J. Pineau, A. Courville, Y. Bengio, A hierarchical latent variable encoder-decoder model for generating dialogues, in: AAAI Conference on Artificial Intelligence, 2017. URL <https://aaai.org/ocs/index.php/AAAI/AAAI17/paper/view/14567>.
O. Uzuner, I. Goldstein, Y. Luo, I. Kohane, Identifying patient smoking status from medical discharge records, J. Am. Med. Inform. Assoc. 15 (1) (2008) 14–24.
I. McCowan, D. Moore, M.-J. Fry, Classification of cancer stage from free-text histology reports, Conf. Proc. IEEE Eng. Med. Biol. Soc. 1 (2006) 5153–5156.
G. Gkotsis, S. Velupillai, A. Oellrich, H. Dean, M. Liakata, R. Dutta, Don’t let notes be misunderstood: a negation detection method for assessing risk of suicide in mental health records, Proceedings of the Third Workshop on Computational Linguistics and Clinical Psychology, Association for Computational Linguistics, San Diego, CA, USA, 2016, pp. 95–105 <http://www.aclweb.org/anthology/W160310>.
H. Kaur, S. Sohn, C.-I. Wi, E. Ryu, M.A. Park, K. Bachman, H. Kita, I. Croghan, J.A. Castro-Rodriguez, G.A. Voge, H. Liu, Y.J. Juhn, Automated chart review utilizing natural language processing algorithm for asthma predictive index, BMC Pulmonary Med. 18 (1) (2018) 34, https://doi.org/10.1186/s12890-018-0593-9.
R. Miotto, L. Li, Brian A. Kidd, Joel T. Dudley, Deep patient: an unsupervised representation to predict the future of patients from the electronic health records, Sci. Rep. 6 (2045-2322) (2016), https://doi.org/10.1038/srep26094.
B. Elvevag, P.W. Foltz, M. Rosenstein, L.E. Delisi, An automated method to analyze language use in patients with schizophrenia and their first-degree relatives, J. Neurolinguist. 23 (3) (2010) 270–284, https://doi.org/10.1016/j.jneuroling.2009.05.002.
C.M. Corcoran, F. Carrillo, D. Fernández-Slezak, G. Bedi, C. Klim, D.C. Javitt, C.E. Bearden, G.A. Cecchi, Prediction of psychosis across protocols and risk cohorts using automated language analysis, World Psychiatry 17 (1) (2018) 67–75, https://doi.org/10.1002/wps.20491 <http://www.ncbi.nlm.nih.gov/pmc/articles/ PMC5775133/>.[67] K.C. Fraser, J.A. Meltzer, N.L. Graham, C. Leonard, G. Hirst, S.E. Black, E. Rochon, Automated classification of primary progressive aphasia subtypes from narrative speech transcripts, Language, Comput. Cognit. Neurosci.55 (2014)43–60, https://doi.org/10.1016/j.cortex.2012.12.006 <http://www.sciencedirect.com/science/article/pii/S0010945212003413>.
[68] E. Keuleers, D.A. Balota, Megastudies, crowdsourcing, and large datasets in psycholinguistics: an overview of recent developments, Quart. J. Exp. Psychol. 68 (8) (2015) 1457–1468, https://doi.org/10.1080/17470218.2015.1051065 pMID:25975773.
[69] G. Coppersmith, M. Dredze, C. Harman, K. Hollingshead, M. Mitchell, CLPsych 2015 Shared Task: Depression and PTSD on Twitter, Proceedings of the 2nd Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality, Association for Computational Linguistics, Denver, Colorado, 2015, pp. 31–39.
[70] A. Benton, M. Mitchell, D. Hovy, Multitask learning for mental health conditions with limited social media data, Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, Long Papers, vol. 1, Association for Computational Linguistics, Valencia, Spain, 2017, pp. 152–162.
[71] A. Tsakalidis, M. Liakata, T. Damoulas, B. Jellinek, W. Guo, A. Cristea, Combining Heterogeneous User Generated Data to Sense Well-being, in: Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, The COLING 2016 Organizing Committee, Osaka, Japan, 2016, pp. 3007–3018.
[72] L. Canzian, M. Musolesi, Trajectories of depression: unobtrusive monitoring of depressive states by means of smartphone mobility traces analysis, Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing, UbiComp ’15, ACM, New York, NY, USA, 2015, pp. 1293–1304,, https://doi.org/10.1145/2750858.2805845.
[73] N. Jaques, S. Taylor, A. Sano, R. Picard, Multi-task, multi-kernel learning for estimating individual wellbeing, in: Proceedings of NIPS Workshop on Multimodal Machine Learning, 2015.
[74] N. Jaques, O. Rudovic, S. Taylor, A. Sano, R. Picard, Predicting tomorrow’s mood, health, and stress level using personalized multitask learning and domain adaptation, in: Proc. IJCAI, 2017.
[75] A. Tsakalidis, M. Liakata, T. Damoulas, A. Cristea, Can we assess mental health through social media and smart devices? Addressing bias in methodology and evaluation, in: Proceedings of ECML-PKDD 2018, the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, The ECML-PKDD Organizing Committee, Dublin, Ireland, 2018.
[76] A.E. Johnson, T.J. Pollard, L. Shen, L.-w. H. Lehman, M. Feng, M. Ghassemi, B. Moody, P. Szolovits, L. Anthony Celi, R.G. Mark, MIMIC-III, a freely accessible critical care database, Sci. Data 3 (2016) 160035.
[77] C.A. Harle, E.H. Golembiewski, K.P. Rahmanian, J.L. Krieger, D. Hagmajer, A.G. Mainous 3rd, R.E. Moseley, Patient preferences toward an interactive e-consent application for research using electronic health records, J. Am. Med. Inform. Assoc. 25 (3) (2018) 360–368, https://doi.org/10.1093/jamia/ocx145 arXiv:/oup/backfile/content_public/journal/jamia/25/3/10.1093_jamia_ocx145/2/ocx145.pdf.
[78] H. Suominen, L. Hanlen, C. Paris, Twitter for health — seeking to understand and curate laypersons’ personal experiences: building a social media search engine to improve search, summarization, and visualization, in: A. Neustein (Ed.), Speech Technology and Natural Language Processing in Medicine and Healthcare, De Gruyter, Berlin, Germany, 2014, pp. 134–174 (Chapter 6).
[79] M. Johnson, S. Lapkin, V. Long, P. Sanchez, H. Suominen, J. Basilakis, L. Dawson, A systematic review of speech recognition technology in health care, BMC Med. Inform. Decis. Mak. 14 (2014) 94.
[80] L. Goeuriot, L. Kelly, H. Suominen, L. Hanlen, A. Névéol, C. Grouin, J.R.M. Palotti, G. Zuccon, Overview of the CLEF eHealth Evaluation Lab 2015, in: J. Mothe, J. Savoy, J. Kamps, K. Pinel-Sauvagnat, G.J.F. Jones, E. SanJuan, L. Cappellato, N. Ferro (Eds.), Experimental IR Meets Multilinguality, Multimodality, and Interaction, Proceedings of the Sixth International Conference of the CLEF Association (CLEF 2015), Lecture Notes in Computer Science (LNCS) 9283, Springer, Heidelberg, Germany, 2015, pp. 429–443.
[81] T. Hodgson, E. Coiera, Risks and benefits of speech recognition for clinical documentation: a systematic review, J. Am. Med. Inform. Assoc. 23 (e1) (2016) e169–e179.
[82] H. Suominen, M. Johnson, L. Zhou, P. Sanchez, R. Sirel, J. Basilakis, L. Hanlen, D. Estival, L. Dawson, B. Kelly, Capturing patient information at nursing shift changes: methodological evaluation of speech recognition and information extraction, J. Am. Med. Inform. Assoc. 22 (e1) (2015) e48–66.
[83] T. Hodgson, F. Magrabi, E. Coiera, Evaluating the usability of speech recognition to create clinical documentation using a commercial electronic health record, Int. J. Med. Inform. 113 (2018) 38–42, https://doi.org/10.1016/j.ijmedinf.2018.02.011 <http://www.sciencedirect.com/science/article/pii/S1386505618300820>.
[84] D. Mollá, B. Hutchinson, Intrinsic versus extrinsic evaluations of parsing systems, Proceedings of the EACL 2003 Workshop on Evaluation Initiatives in Natural Language Processing: Are Evaluation Methods, Metrics and Resources Reusable?, Evalinitiatives ’03, Association for Computational Linguistics, Stroudsburg, PA, USA, 2003, pp. 43–50 <http://dl.acm.org/citation.cfm?id=1641396.1641403>.
[85] K. Nguyen, B. O’Connor, Posterior calibration and exploratory analysis for natural language processing models, Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, Lisbon, Portugal, 2015, pp. 1587–1598 <http://aclweb.org/anthology/D15-1182>.
[86] W. Scuba, M. Tharp, D. Mowery, E. Tseytlin, Y. Liu, F.A. Drews, W.W. Chapman, Knowledge Author: facilitating user-driven, domain content development to support clinical information extraction, J. Biomed. Semant. 7 (1) (2016) 42, https://doi.org/10.1186/s13326-016-0086-9.
[87] J.P.A. Ioannidis, Why most clinical research is not useful, PLOS Med. 13 (6) (2016) 1–10, https://doi.org/10.1371/journal.pmed.1002049.
[88] J.P.A. Ioannidis, How to make more published research true, PLoS Med. 11 (10) (2014) e1001747, https://doi.org/10.1371/journal.pmed.1001747.
[89] E. von Elm, D.G. Altman, M. Egger, S.J. Pocock, P.C. Gotzsche, J.P. Vandenbroucke, The strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies, Lancet (London, England) 370 (9596) (2007) 1453–1457, https://doi.org/10.1016/S0140-6736(07)61602-X.
[90] E.I. Benchimol, L. Smeeth, A. Guttmann, K. Harron, D. Moher, I. Petersen, H.T. Sørensen, E. von Elm, S.M. Langan, R.W. Committee, The reporting of studies conducted using observational routinely-collected health data (record) statement, PLOS Med. 12 (10) (2015) 1–22, https://doi.org/10.1371/journal.pmed.1001885.
[91] R. Gilbert, R. Lafferty, G. Hagger-Johnson, K. Harron, L.-C. Zhang, P. Smith, C. Dibben, H. Goldstein, Guild: Guidance for information about linking data sets, J. Public Health 40 (1) (2018) 191–198, https://doi.org/10.1093/pubmed/fdx037 arXiv:/oup/backfile/content_public/journal/jpubhealth/40/1/10.1093_pubmed_fdx037/3/fdx037.pdf.

Full Text

Your rating :

Using Clinical Natural Language Processing for Health Outcomes Research: Overview and Actionable Suggestions for Future Advances

Introduction:

Methods:

Results:

Conclusion: