Document Type


Publication Date


Publication Title

Journal of affective disorders


BACKGROUND: Few studies report on machine learning models for suicide risk prediction in adolescents and their utility in identifying those in need of further evaluation. This study examined whether a model trained and validated using data from all age groups works as well for adolescents or whether it could be improved.

METHODS: We used healthcare data for 1.4 million specialty mental health and primary care outpatient visits among 256,823 adolescents across 7 health systems. The prediction target was 90-day risk of suicide attempt following a visit. We used logistic regression with least absolute shrinkage and selection operator (LASSO) and generalized estimating equations (GEE) to predict risk. We compared performance of three models: an existing model, a recalibrated version of that model, and a newly-learned model. Models were compared using area under the receiver operating curve (AUC), sensitivity, specificity, positive predictive value and negative predictive value.

RESULTS: The AUC produced by the existing model for specialty mental health visits estimated in adolescents alone (0.796; [0.789, 0.802]) was not significantly different than the AUC of the recalibrated existing model (0.794; [0.787, 0.80]) or the newly-learned model (0.795; [0.789, 0.801]). Predicted risk following primary care visits was also similar: existing (0.855; [0.844, 0.866]), recalibrated (0.85 [0.839, 0.862]), newly-learned (0.842, [0.829, 0.854]).

LIMITATIONS: The models did not incorporate non-healthcare risk factors. The models relied on ICD9-CM codes for diagnoses and outcome measurement.

CONCLUSIONS: Prediction models already in operational use by health systems can be reliably employed for identifying adolescents in need of further evaluation.

PubMed ID




First Page


Last Page




To view the content in your browser, please download Adobe Reader or, alternately,
you may Download the file to your hard drive.

NOTE: The latest versions of Adobe Reader do not support viewing PDF files within Firefox on Mac OS and if you are using a modern (Intel) Mac, there is no official plugin for viewing PDF files within the browser window.