Exploring the Hyperparameters of XGBoost Through 3D Visualizations.

Optimizing the hyperparameters is one of the most important and time-consuming activities to do when training machine learning models. But the lack of guidance available to optimization algorithms means that finding values for these hyperparameters is left to black-box methods. Black-box methods can be made more efficient by incorporating an understanding of where good hyperparameter values might be located for a specific model. In this paper, we visualize hyperparameter performance-landscapes in several datasets to discover how the XGBoost algorithm behaves for many combinations of hyperparameter values across these datasets. Using this knowledge, it might be possible to design more efficient search strategies for optimizing the hyperparameters of XGBoost.