Comparative Evaluation of AI-Based Systems for Tinnitus

Abstract

Abstract Introduction: Today, with the development of technology, the variety of information sources has increased. It is now possible to access information obtained from encyclopedias in seconds with a few clicks of a button. Rapid developments in artificial in telligence (AI) and the widespread use of large language models (LLMs) such as ChatGPT, Gemini, and Perplexity have revolutionized access to medi cal information. However, the accuracy and readability of the answers provided by these models are critical, especially in the healthcare domain. This study evaluates the performance of ChatGPT, Gemini, and Perplexity in addressing frequently asked questions abou t tinnitus, a common symptom in otolaryngology practice. Materials and Methods: Twenty frequently asked questions about tinnitus were posed to the models and their responses were evaluated by two otolaryngologists using global quality (GQS) and Likert scales for accuracy and reliability and the Gunni ng-Fog Index (GFI) for readability. Results: The findings reveal no significant difference in the reliability and quality of information between the models, but it was ob served that Gemini came out ahead in readability and ChatGPT in accuracy. However, Perplexity lagged in both metrics. These results highlight the varying strengths and weaknesses of LLMs, emphasizing the importance of model selection based on user needs. For example, ChatGPT is ideal for complex medical information, while Gemini is more acc essible to wider audiences. Conclusion: This study demonstrates the potential of AI-enabled systems in healthcare; however, we suggest that future improvements should increase both accuracy and accessibility.