Evaluating AI Text Detection Tools for Distinguishing Human-Written from AI-Generated Abstracts in Persian-Language Journals of Library and Information Science
Evaluating AI Text Detection Tools for Distinguishing Human-Written from AI-Generated Abstracts in Persian-Language Journals of Library and Information Science

Summary/Abstract: Background: Researchers are using artificial intelligence (AI) tools in academic writing. However, their use may compromise the integrity and originality of the work. Hence, AI text detection tools have come to increase transparency. Objective: This study aims to evaluate the accuracy of AI text detection tools in recognizing human-written and AI-written abstracts in library and information science (LIS). Methods: Seven Persian academic journals in LIS were selected. ZeroGPT and GPT Zero as AI text detectors were used. AI-generated abstracts were produced by AI chatbots (ChatGPT 4.0, DeepSeek and Qwen). Results: Despite performing strongly in detecting AI-generated text, especially from models such as DeepSeek and Qwen, ZeroGPT and GPTZero struggle to accurately identify human-written content, resulting in high false positive rates and raising concerns about their reliability. Conclusion: The findings highlight the need for culturally and linguistically inclusive AI detection tools, as current systems such as ZeroGPT and GPTZero show limitations in diverse language contexts, underscoring the importance of improved algorithms and human-involved evaluation to ensure fairness and reliability in academic settings.

• Details
• Contents