A Comparative Study of AI-generated and Teacher-constructed Language Tests
A Comparative Study of AI-generated and Teacher-constructed Language Tests

Summary/Abstract: This article examines the quality of multiple-choice (MC) vocabulary tests generated with the help of artificial intelligence (AI) by comparing their psychometric properties to those of human-constructed tests. Two sets of criterion-referenced tests (CRTs), designed to assess vocabulary previously taught in class, were developed. In each set, one test was entirely generated by AI, while the other incorporated MC options either fully created by a human constructor or modified from the AI-generated options. The tests were administered to students of a high school. The analysis focussed on reliability estimates and item statistics, particularly those which are relevant to CRTs. The findings suggest that the use of AI significantly improved test practicality by reducing the time and effort needed to develop the tests, although human-constructed tests exhibited superior psychometric qualities.

• Details
• Contents