Comparative Evaluation of ChatGPT-4 andOphthalmologist-in-training in the Triage ofPatient Messages Sent to the Eye Clinic viaElectronic Medical Record

Document Type

Conference Proceeding

Publication Date

6-1-2025

Publication Title

Invest Ophthalmol Vis Sci

Keywords

alanine aminotransferase, ChatGPT, cohort analysis, conference abstract, controlled study, drug therapy, female, generative pretrained transformer, human, major clinical study, male, medical record, ophthalmologist, patient triage, training, workload

Abstract

Purpose : Chat Generative Pre-Trained Transformer (GPT-4), an artificial Intelligencemodel, has been proposed to have many applications in the field of medicine includingdiagnosis, triage, medical recordkeeping, education, and literature analysis. The role ofGPT-4 in Ophthalmology has previously been studied only to process queries posted in anonline medical forum. We conducted a restrospective cohort study to test theeffectiveness of GPT-4 in triaging real patient messages sent to our eye clinic. Methods : Messages from patients sent via Epic MyChart to Virginia CommonwealthUniversity (VCU) General Eye Clinic from January 2023 - August 2023 were recorded.Patient messages sent to the general eye clinic were de-identified and triaged by anophthalmologist-in-training as well as GPT-4. Ophthalmologists and GPT-4 were bothasked to direct patients to either general or specialty eye clinics, urgently or non-urgently,depending on the severity of the condition. Our two main outcome measures includedGPT-4's ability to accurately direct patient messages to 1) a general or specialty eye clinicand 2) determine the time frame within which the patient needed to be seen (triageacuity). Accuracy of recommendations made by GPT-4 were determined by comparingpercent agreement with those made by the Ophthalmologist.Results : The study included 139 patient messages. We noted agreement betweenrecommendations made by ChatGPT and Ophthalmologist in the majority of the cases.Similar recommendations were made with regard to general/specialty clinic (64.7%) andtriage acuity (60.4%). We also noted that GPT-4 recommended a triage acuity equal to orsooner than ophthalmologist for 93.5% of cases, and only recommended a less urgenttriage acuity in 6.5% of cases.Conclusions : To our knowledge, this is the first time that GPT-4 was assessed using realpatient queries in an Ophthalmology clinic. Our study indicates an Artifical Intelligencesystem such as GPT-4 would complement physician judgment in triaging ophthalmiccomplaints. These systems may assist providers and reduce the workload ofophthalmologists and ophthalmic .

Volume

66

Issue

8

Find It @ Sladen

Share

COinS