Effect of Dimension Size and Window Size on Word Embedding in Classification Tasks
Effect of Dimension Size and Window Size on Word Embedding in Classification Tasks

Summary/Abstract: Background: Static word embedding models such as Word2Vec and GloVe remain widely used in natural language processing, yet key hyperparameters are often selected heuristically rather than through systematic validation. Objective: This study provides an extrinsic evaluation of context window size and embedding dimensionality for Word2Vec (CBOW and Skip-gram) and GloVe embeddings in a downstream spam classification task. Methods: Embeddings were trained on a large external corpus and evaluated using a neural network and several classical machine learning classifiers. Results: The results show that context window size has a moderate influence on performance, whereas embedding dimensionality has a clearer effect: values below approximately 50 degrade performance, while increases beyond moderate ranges (approximately 100–150) yield diminishing returns. Across all experiments, Word2Vec achieves higher stability and performance than GloVe. Conclusion: Overall, the findings suggest that robust classification performance can be achieved with moderate embedding dimensionalities and smaller context windows, providing practical guidance for efficient embedding configuration.

• Details
• Contents