It is sometimes necessary to consider more general nonlinear programs than the ones we have already studied: linear, quadratic, or conic programs. We give a brief introduction to this vast topic, and we discuss an application to estimating a volatility surface.

Nonlinear Programming

Consider a very general optimization problem of the form

or the equivalent more concise form

where. In the special case when all functions, hi are linear, problem (20.1) is a linear program as discussed in Chapter 2. When some of the functions, are nonlinear, problem (20.1) is a nonlinear program.

Many practical problems are naturally formulated as nonlinear programs. We already saw quadratic programming and conic programming in earlier chapters. However, the family of problems that can be formulated as nonlinear programs is enormous, as many, if not most, imaginable kinds of constraints and objectives can be cast in terms of nonlinear functions. (For some noteworthy examples, see the exercises at the end of the chapter.) The immense modeling power of nonlinear programming comes at a cost: unlike linear, quadratic, and conic programming, which have a solid theory and are solvable via a few algorithmic templates, both the theory and methods to solve general nonlinear programs are far more complicated. Different types of nonlinear programs, determined by structural properties of the objective and constraint functions, are amenable to different types of algorithms. The subsequent sections sketch the main theory and most popular algorithmic ideas. A comprehensive treatment of this vast topic is beyond the scope of this textbook. We refer the reader to the excellent references Bertsekas (1999), Guler (2010), and Nocedal and Wright (2006) for more details.

Numerical Nonlinear Programming Solvers

There are numerous software packages for solving nonlinear programs. The following are some popular ones. We list them according to the class of algorithms (discussed in Section 20.4) that they are based on: