YAPAY ZEKA TABANLI OTOMATİK PUANLAMA SİSTEMLERİNİN GÜVENİRLİK VE GEÇERLİK AÇISINDAN DEĞERLENDİRİLMESİ
EVALUATION OF ARTIFICIAL INTELLIGENCE BASED AUTOMATED SCORING SYSTEMS IN TERMS OF RELIABILITY AND VALIDITY

Summary/Abstract: With the rapid development of educational technologies today, the integration of artificial intelligence (AI)-based systems into assessment and evaluation processes has come to the forefront. Particularly in largescale exams and the evaluation of open-ended questions, AI-based automated scoring systems offer advantages in terms of time, cost, and labor. These systems have the potential to enhance objectivity and standardize processes in education. However, there is ongoing debate about the reliability (consistency) and validity (accuracy) levels of these systems when compared to traditional human raters. This study aims to evaluate the effectiveness of AI-based automated scoring systems in education in terms of reliability and validity. Conducted based on a review of the literature, the study examines national and international master's theses, scholarly articles, and policy documents to provide a comprehensive analysis of these systems. This includes comparative analyses with human scorers, levels of internal consistency, alignment with rubric criteria, user perceptions, and the future of the technology in education. The findings indicate that while AIsupported scoring systems have not yet fully replaced human judgment, they do offer high levels of consistency and speed under certain conditions. Moreover, it has been observed that teacher and student perceptions affect the acceptability of the system, and demographic variables create significant differences in these perceptions.

• Details
• Contents