Evaluating LLMs as Educational Co-authors: A Methodological Framework
Evaluating LLMs as Educational Co-authors: A Methodological Framework

Author(s): Bruno Polonijo, Sabrina Suman, Tomislav Car
Subject(s): Social Sciences
Published by: Udruženje ekonomista i menadžera Balkana
Keywords: Methodological framework; Evaluation; LLMs; Educational; Co-authors; AI; Prompt engineering
Summary/Abstract: This paper introduces and applies a novel methodological framework for the comprehensive evaluation of Large Language Models (LLMs) as educational co-authors. The purpose of this work is to address the lack of structured, reproducible processes for assessing the quality and practical utility of AI-generated educational content. A controlled experiment was conducted in which a standardised task was assigned to several leading LLMs. The effectiveness and quality of each model were evaluated within a newly developed framework that measures both the final product quality and the “refinement effort” required. The results reveal a non-linear evolution in model capabilities, highlighting persistent shortcomings in complex reasoning. The findings emphasize the importance of a holistic evaluation that considers both the final output and the efficiency of the collaborative process, offering a practical tool for educators and researchers.

• Details
• Contents