Many problems that we have seen already require that we “minimize” a certain function, or find the “minimum” of a function of many variables. Formally, the problem is: given F(x), find a value x0 such that F(x) > F(x0) for all x, for a global minimum, or F(x) > F(x0) where x ϵ [a, b] (some constrained range) for a local minimum. We will review these familiar problems, and introduce some new ones. Our first job will then be to write a function that can find the minimum of a function of one variable, a sort of minimization analogue to the bisection method from Section 3.2.1 for finding roots of F(x). Then we will study methods that can be used to minimize a function u(x) where x ϵ ℝn.

Physical motivation

There are many places in physics where finding the minimum of a function can be useful. We have already done linear minimization in solving the least squares problem for data fitting from Section 10.2 – there we turned the minimization of the Pythagorean length of the residual vector into a matrix inverse problem with a unique solution. There are nonlinear extensions of this process that rely on iterative techniques directly (as opposed to the matrix inversion that solves the least squares problem, which can be performed iteratively or not).

Recall the harmonic approximation from Chapter 8: Given the potential energy U(x), some complicated function, we know that in the vicinity of a local minimum, the motion of a particle will be harmonic with effective spring constant set by the second derivative of the potential evaluated at the minimum.