Design and Analysis of Machine Learning Experiments

This chapter contains sections titled: 19.1 Introduction, 19.2 Factors, Response, and Strategy of Experimentation, 19.3 Response Surface Design, 19.4 Randomization, Replication, and Blocking, 19.5 Guidelines for Machine Learning Experiments, 19.6 Cross-Validation and Resampling Methods, 19.7 Measuring Classifier Performance, 19.8 Interval Estimation, 19.9 Hypothesis Testing, 19.10 Assessing a Classification Algorithm's Performance, 19.11 Comparing Two Classification Algorithms, 19.12 Comparing Multiple Algorithms: Analysis of Variance, 19.13 Comparison over Multiple Datasets, 19.14 Notes, 19.15 Exercises, 19.16 References