In this book, we are going to examine various types of numerical problems and develop techniques for solving them. We will gradually build up the necessary tools and use a number of case studies to:

1. illustrate the process of formulating a problem, by which we mean translating from an intuitive idea of the problem by writing it down mathematically,

2. motivate and develop algorithms to solve problems, that is, descriptions of operations that can be implemented in software to take a problem specification and return a solution, and

3. illustrate how to match the formulation of the problem to the capabilities of available algorithms, involving, in some cases, transformation of the problem from its initial formulation.

We illustrate how to think about and describe problems to make them amenable to solution by optimization software. In brief: formulation of problems to facilitate their solution.

In the five parts of this book, we will consider five general problem classes. The first problem class is the solution of linear systems of equations (Part I). Solving linear systems of equations will turn out to be at the center of much of our subsequent work. For example, our second problem class, solving non-linear systems of equations (Part II), will be solved by an algorithm that requires repeated solution of linear systems of equations.

We will then go on to use these two algorithms as building blocks in algorithms for:

• unconstrained optimization (Part III), which will in turn form part of the algorithms for:

• equality-constrained optimization (Part IV), which will again form part of the algorithms for:

• inequality-constrained optimization (Part V).