Book contents
- Frontmatter
- Dedication
- Contents
- Preface
- Notation
- Part One Machine Learning
- Executive Summary
- 1 Rudiments of Statistical Learning Theory
- 2 Vapnik–Chervonenkis Dimension
- 3 Learnability for Binary Classification
- 4 Support Vector Machines
- 5 Reproducing Kernel Hilbert Spaces
- 6 Regression and Regularization
- 7 Clustering
- 8 Dimension Reduction
- Part Two Optimal Recovery
- Part Three Compressive Sensing
- Part Four Optimization
- Part Five Neural Networks
- Appendices
- References
- Index
6 - Regression and Regularization
from Part One - Machine Learning
Published online by Cambridge University Press: 21 April 2022
- Frontmatter
- Dedication
- Contents
- Preface
- Notation
- Part One Machine Learning
- Executive Summary
- 1 Rudiments of Statistical Learning Theory
- 2 Vapnik–Chervonenkis Dimension
- 3 Learnability for Binary Classification
- 4 Support Vector Machines
- 5 Reproducing Kernel Hilbert Spaces
- 6 Regression and Regularization
- 7 Clustering
- 8 Dimension Reduction
- Part Two Optimal Recovery
- Part Three Compressive Sensing
- Part Four Optimization
- Part Five Neural Networks
- Appendices
- References
- Index
Summary
This chapter switches from classification to regression. Concentrating on the square loss, it explains how empirical risk minimization becomes a least-squares problem, with different characteristics in the underparametrized regime and in the overparametrized regime. Adding a regularizer to the empirical risk is common in the latter regime, and the examples of Tikhonov regularization and of LASSO are discussed. The chapter concludes by highlighting a way of interpreting classification problems as regularization problems.
- Type
- Chapter
- Information
- Mathematical Pictures at a Data Science Exhibition , pp. 41 - 46Publisher: Cambridge University PressPrint publication year: 2022