Developing Machine Learning Models for Behavioral Coding
Recommended Citation
Idalski Carcone A, Hasan M, Alexander GL, Dong M, Eggly S, Brogan Hartlieb K, Naar S, MacDonell K, and Kotov A. Developing Machine Learning Models for Behavioral Coding. J Pediatr Psychol 2019; 44(3):289-299.
Document Type
Article
Publication Date
1-29-2019
Publication Title
Journal of pediatric psychology
Abstract
Objective: The goal of this research is to develop a machine learning supervised classification model to automatically code clinical encounter transcripts using a behavioral code scheme.
Methods: We first evaluated the efficacy of eight state-of-the-art machine learning classification models to recognize patient-provider communication behaviors operationalized by the motivational interviewing framework. Data were collected during the course of a single weight loss intervention session with 37 African American adolescents and their caregivers. We then tested the transferability of the model to a novel treatment context, 80 patient-provider interactions during routine human immunodeficiency virus (HIV) clinic visits.
Results: Of the eight models tested, the support vector machine model demonstrated the best performance, achieving a .680 F1-score (a function of model precision and recall) in adolescent and .639 in caregiver sessions. Adding semantic and contextual features improved accuracy with 75.1% of utterances in adolescent and 73.8% in caregiver sessions correctly coded. With no modification, the model correctly classified 72.0% of patient-provider utterances in HIV clinical encounters with reliability comparable to human coders (k = .639).
Conclusions: The development of a validated approach for automatic behavioral coding offers an efficient alternative to traditional, resource-intensive methods with the potential to dramatically accelerate the pace of outcomes-oriented behavioral research. The knowledge gained from computer-driven behavioral research can inform clinical practice by providing clinicians with empirically supported communication strategies to tailor their conversations with patients. Lastly, automatic behavioral coding is a critical first step toward fully automated eHealth/mHealth (electronic/mobile Health) behavioral interventions.
PubMed ID
30698755
Volume
44
Issue
3
First Page
289
Last Page
299
