Building Ensemble Models with Python

Summary

This chapter uses several available Python packages to build predictive models using the ensemble algorithms. It demonstrates ensemble methods available as XGBoost, Python sklearn, and PySpark packages. The chapter also covers regression, binary classification, and multiclass classification problems, and discusses variations on these themes such as the workings of coding categorical variables for input to XGBoost, Python, and PySpark ensemble methods and stratified sampling. Ensemble methods are relatively easy to use—they do not have many parameters to tune, they give variable importance data to help in the early stages of model development, and they very often give the best performance achievable. Non‐numeric attributes can improve prediction accuracy, but Python ensemble methods need numeric input. The Python scikit‐learn ensemble module houses a random forest algorithm for regression problems. The gradient boosting and random forest packages in XGBoost and the Python sklearn library will build both binary and multiclass classification models.