JAMA Ophthalmology

中文摘要

這篇研究報告評估了人工智能（AI）聊天機器人ChatGPT在眼科知識評估中的表現。研究使用了由OphthoQuestions提供的眼科認證考試準備的多選題作為資料，並分析了ChatGPT在2023年1月9日至16日和2月17日回答問題的情況。研究的主要結果是ChatGPT正確回答的認證考試練習題數量，次要結果包括ChatGPT提供額外解釋的問題比例、ChatGPT回答問題和回應的平均長度、ChatGPT回答沒有多選選項的問題的表現，以及表現的時間變化。結果顯示，ChatGPT在OphthoQuestions眼科認證考試準備的免費試用中大約正確回答了一半的問題。然而，該研究指出，在目前的情況下，ChatGPT的表現還不足以在認證考試的準備中提供實質性的幫助。作者強調了醫學專業人士應該重視人工智能在醫學領域的進展，同時也應該認識到ChatGPT在這項研究中的表現還不足以提供充分的協助。

English Abstract

This study assessed the performance of an artificial intelligence (AI) chatbot called ChatGPT in answering ophthalmic knowledge test questions. The study used a sample of text-based multiple-choice questions from a practice question bank. The primary outcome was the number of questions that ChatGPT answered correctly. The secondary outcomes included the proportion of questions for which ChatGPT provided explanations, the length of questions and responses, performance in answering questions without multiple-choice options, and changes in performance over time. The study found that ChatGPT answered approximately half of the questions correctly (58 of 125 questions, 46% in Jan, 2023 and 73 of 125 questions, 58% in Feb, 2023), and provided explanations for most questions. However, its performance was not sufficient to provide substantial assistance in preparing for board certification exams. The authors emphasized the importance of responsible use of AI systems in medical education and clinical practice.