Book contents
- Frontmatter
- Dedication
- Contents
- Preface
- Notation
- Part One Machine Learning
- Part Two Optimal Recovery
- Part Three Compressive Sensing
- Part Four Optimization
- Executive Summary
- 19 Basic Convex Optimization
- 20 Snippets of Linear Programming
- 21 Duality Theory and Practice
- 22 Semidefinite Programming in Action
- 23 Instances of Nonconvex Optimization
- Part Five Neural Networks
- Appendices
- References
- Index
19 - Basic Convex Optimization
from Part Four - Optimization
Published online by Cambridge University Press: 21 April 2022
- Frontmatter
- Dedication
- Contents
- Preface
- Notation
- Part One Machine Learning
- Part Two Optimal Recovery
- Part Three Compressive Sensing
- Part Four Optimization
- Executive Summary
- 19 Basic Convex Optimization
- 20 Snippets of Linear Programming
- 21 Duality Theory and Practice
- 22 Semidefinite Programming in Action
- 23 Instances of Nonconvex Optimization
- Part Five Neural Networks
- Appendices
- References
- Index
Summary
This chapter introduces the key concepts of optimization, such as objective function, constraints, local and global minimizers, and gradient descent algorithms. The rate of convergence for the steepest descent algorithm is analyzed when the objective function is smooth and convex or smooth and strongly convex. The analysis is extended to the stochastic gradient descent algorithm.
- Type
- Chapter
- Information
- Mathematical Pictures at a Data Science Exhibition , pp. 160 - 168Publisher: Cambridge University PressPrint publication year: 2022