Explainable Artificial Intelligence Tools as a Part of the Best Practices in Model Selection in Business Decision Modelling. Example of Marketing Campaign Success Forecasting
Explainable Artificial Intelligence Tools as a Part of the Best Practices in Model Selection in Business Decision Modelling. Example of Marketing Campaign Success Forecasting

Author(s): Marcin Chlebus, Zuzanna Osika
Subject(s): Business Economy / Management, Marketing / Advertising, ICT Information and Communications Technologies
Published by: Wydawnictwa Uniwersytetu Warszawskiego
Keywords: business decision models; machine learning; bank marketing; explainable artificial intelligence (XAI)
Summary/Abstract: Complex machine learning techniques are increasingly used in business decision-support models. Typically, model evaluation relies solely on predictive or discriminatory power. However, numerous empirical examples demonstrate that relying on such single-dimensional criteria can result in significant failures. This paper proposes incorporating Explainable Artificial Intelligence (XAI) methods to enhance model evaluation. Specifically, we demonstrate how global XAI techniques – Permutation Feature Importance (PFI) and Partial Dependence Plots (PDP) – can be integrated into the model assessment process. In our study, we compared tree-based algorithms for predicting the success of a Portuguese bank’s telemarketing campaign. Five models were tested: Random Forest, AdaBoost, GBM, XGBoost, and CatBoost. The top three in terms of AUC were XGBoost, CatBoost, and GBM, with XGBoost performing best. These models were then further analysed using PDP and PFI, which revealed potential overfitting in XGBoost and GBM, leading to CatBoost being selected as the most reliable model. The findings suggest that relying on predictive performance alone may not guarantee the optimal model choice. Expanding evaluation with XAI methods provides deeper insights and supports more robust model selection.

• Details
• Contents