Evaluating Transformer-Based Foundation Models for Time-Series Forecasting Across Multiple Horizons
Evaluating Transformer-Based Foundation Models for Time-Series Forecasting Across Multiple Horizons

Summary/Abstract: Recent studies show that transformer-based architectures and foundation models have achieved promising results in time-series forecasting, yet their advantages over traditional machine-learning methods remain inconsistent across domains and forecasting horizons. This study investigates whether modern transformer-based models provide systematic performance gains compared to classical regression approaches and examines four representative models as SAMFormer, TimesFM, Time-MoE, and TimeGPT and benchmark them against established baselines including Linear Regression, MLPRegressor and ensemble methods. In the paper it is hypothesized that transformer-based models outperform traditional methods, particularly for longer forecasting horizons and datasets with strong temporal dependencies. An experimental evaluation is conducted across four real-world datasets from weather, finance, energy, and healthcare domains, using multiple context and prediction length settings. Model performance is assessed using standard error-based and distribution-based metrics. The results show that transformer-based models generally outperform regression baselines, with SAMFormer demonstrating the most stable and consistent performance across datasets and horizons. TimeGPT excels in short-term forecasting, while TimesFM exhibits limited robustness, especially for longer horizons. Fine-tuning yields mixed benefits, depending on the dataset and model architecture. Overall, the findings provide an evidence-based assessment of when transformer-based forecasting is advantageous and when simpler models remain competitive, offering practical guidance for model selection in real-world time-series applications.

• Details
• Contents