Comparative Evaluation of ChatGPT-4 andOphthalmologist-in-training in the Triage ofPatient Messages Sent to the Eye Clinic viaElectronic Medical Record
Recommended Citation
Deshmukh S, Alsumait A, Wang C, Leffler C. Comparative Evaluation of ChatGPT-4 andOphthalmologist-in-training in the Triage ofPatient Messages Sent to the Eye Clinic viaElectronic Medical Record. Invest Ophthalmol Vis Sci 2025; 66(8).
Document Type
Conference Proceeding
Publication Date
6-1-2025
Publication Title
Invest Ophthalmol Vis Sci
Abstract
Purpose : Chat Generative Pre-Trained Transformer (GPT-4), an artificial Intelligencemodel, has been proposed to have many applications in the field of medicine includingdiagnosis, triage, medical recordkeeping, education, and literature analysis. The role ofGPT-4 in Ophthalmology has previously been studied only to process queries posted in anonline medical forum. We conducted a restrospective cohort study to test theeffectiveness of GPT-4 in triaging real patient messages sent to our eye clinic. Methods : Messages from patients sent via Epic MyChart to Virginia CommonwealthUniversity (VCU) General Eye Clinic from January 2023 - August 2023 were recorded.Patient messages sent to the general eye clinic were de-identified and triaged by anophthalmologist-in-training as well as GPT-4. Ophthalmologists and GPT-4 were bothasked to direct patients to either general or specialty eye clinics, urgently or non-urgently,depending on the severity of the condition. Our two main outcome measures includedGPT-4's ability to accurately direct patient messages to 1) a general or specialty eye clinicand 2) determine the time frame within which the patient needed to be seen (triageacuity). Accuracy of recommendations made by GPT-4 were determined by comparingpercent agreement with those made by the Ophthalmologist.Results : The study included 139 patient messages. We noted agreement betweenrecommendations made by ChatGPT and Ophthalmologist in the majority of the cases.Similar recommendations were made with regard to general/specialty clinic (64.7%) andtriage acuity (60.4%). We also noted that GPT-4 recommended a triage acuity equal to orsooner than ophthalmologist for 93.5% of cases, and only recommended a less urgenttriage acuity in 6.5% of cases.Conclusions : To our knowledge, this is the first time that GPT-4 was assessed using realpatient queries in an Ophthalmology clinic. Our study indicates an Artifical Intelligencesystem such as GPT-4 would complement physician judgment in triaging ophthalmiccomplaints. These systems may assist providers and reduce the workload ofophthalmologists and ophthalmic .
Volume
66
Issue
8
