Performance of an Artificial Intelligence Chatbot in Ophthalmic Knowledge Assessment


Journal JAMA Ophthalmology
Status 審查完成
校稿者 蕭靜熹 醫師

JAMA Ophthalmology



English Abstract

This study assessed the performance of an artificial intelligence (AI) chatbot called ChatGPT in answering ophthalmic knowledge test questions. The study used a sample of text-based multiple-choice questions from a practice question bank. The primary outcome was the number of questions that ChatGPT answered correctly. The secondary outcomes included the proportion of questions for which ChatGPT provided explanations, the length of questions and responses, performance in answering questions without multiple-choice options, and changes in performance over time. The study found that ChatGPT answered approximately half of the questions correctly (58 of 125 questions, 46% in Jan, 2023 and 73 of 125 questions, 58% in Feb, 2023), and provided explanations for most questions. However, its performance was not sufficient to provide substantial assistance in preparing for board certification exams. The authors emphasized the importance of responsible use of AI systems in medical education and clinical practice.