Financial Question-Answering Dataset for Slovak Language Model Evaluation
Financial Question-Answering Dataset for Slovak Language Model Evaluation
Author(s): Daniel Hládek, Kristián Sopkovič, Ján Staš, Zuzana Sokolová, Matúš PlevaSubject(s): Language studies, Applied Linguistics, Computational linguistics, Western Slavic Languages, Financial Markets, ICT Information and Communications Technologies
Published by: SAV - Slovenská akadémia vied - Jazykovedný ústav Ľudovíta Štúra Slovenskej akadémie vied
Keywords: question answering; financial domain; large language model; evaluation; Slovak language resource;
Summary/Abstract: The limited availability of language resources for Slovak presents a significant challenge for the development and evaluation of language models. In this paper, we introduce a multiple-choice question-answering dataset specifically designed for the financial domain in Slovak. The dataset contains 1,334 questions, each with one correct answer and four incorrect ones. It is systematically organized by topic and difficulty level to facilitate structured evaluation. Using this dataset, we assess the performance of several Slovak generative language models and compare their results against a general question-answering dataset to analyze domain-specific model capabilities. The best-performing model is a monolingual Slovak model. Furthermore, the observed performance differences between financial-domain and general question-answering tasks suggest that domain-specific language modeling requires further research.
Journal: Jazykovedný časopis
- Issue Year: 76/2025
- Issue No: 1
- Page Range: 247-257
- Page Count: 11
- Language: English
