Comparison of artificial intelligence systems in answering prosthodontics questions from the dental specialty exam in Turkey

Background/purpose: Artificial intelligence (AI) is increasingly vital in dentistry, supporting diagnostics, treatment planning, and patient education. However, AI systems face challenges, especially in delivering accurate information within specialized dental fields. This study aimed to evaluate the performance of seven AI-based chatbots (ChatGPT-3.5, ChatGPT-4, Gemini, Gemini Advanced, Claude AI, Microsoft Copilot, and Smo din AI) in correctly answering prosthodontics questions from the Dental Specialty Exam (DUS) in Turkey. Materials and methods: The dataset for this study consists of 128 multiple-choice prosthodontics questions from the DUS, a national exam administered in Turkey by the Student Selection and Placement Center (O categorizing the questions into case-based and knowledge-based. Results: ChatGPT-4 achieved the highest accuracy (75.8 %), while Gemini AI had the lowest (46.1 %). Gemini AI also had more incorrect (69) than correct answers (59). ChatGPT-4 and ChatGPT-3.5 showed significantly higher accuracy in knowledge-based questions compared to case-based ones (p < 0.05). For case-based questions, Gemini and Gemini Advanced had the lowest accuracy (36.4 %), while other chatbots averaged 45.5 %. In knowledge-based questions, ChatGPT-4 performed best (78.6 %) and Gemini AI the worst (47 %). Conclusion: ChatGPT-4 excelled in knowledge-based prosthodontic questions, showing potential to enhance dental education through personalized learning and clinical reasoning support. However, its limitations in case-based scenarios highlight the need for optimization to better address complex clinical situations. These findings suggest that AI models can significantly contribute to dental education and clinical practice. (c) 2025 Association for Dental Sciences of the Republic of China. Publishing services by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons. org/licenses/by-nc-nd/4.0/). <spacing diaeresis> SYM) between 2012 and 2021. Chatbot performance was assessed by

Journal

JOURNAL OF DENTAL SCIENCES

Year

2025

Volume

20

Issue

3

Pages

1454-1459

ISSN

1991-7902

eISSN

2213-8862

DOI

10.1016/j.jds.2025.01.025

WoS UID

WOS:001528958200008