Introduction

In 1714-1715, the renowned Japanese Confucian scholar Ogyū Sorai 荻生徂 徠 (1666-1728) published Yakubun sentei 譯文筌蹄 (A Tool for Translation, editor's note 1711), an aid for Sinitic studies. In the ‘Prefatory Remarks in Ten Principles’ (Daigen jussoku 題言十則) to this work, Sorai offered his views on ‘Chinese language’ (Chūka gengo 中華言語):

What did Sorai mean by ‘Chūka gengo’? Both the term gen and the term go suggest a spoken component. The character gen 言 (Ch. yan) stresses spoken words and go 語 (Ch. yu) means words to state or to conversate. The supposed written traces of the vanished spoken words are also called gen or go. By using gengo rather than bun 文 (Ch. wen; pattern, writing, composition), Sorai was emphasizing speech rather than writing.

The chapter focuses on network flow problems, which form a very important part of practical applications. Routing, distribution, and scheduling problems often belong to this category of formulations, while a large number of other optimization problems encountered in diverse areas of applications may contain elements of network flow problems.

Quadratic multidimensional functions play a very important role in the understanding of general nonlinear functions. Convexity of quadratic functions is linked in a natural way from its geometrical definition all the way to the properties of its matrix eigenspectrum. Indeed, to second order expansion, and close to the expansion point, any nonlinear function can be approximated by a quadratic – thus providing a crucial link and understanding of the local behaviour and convexity properties of general functions.

The chapter introduces basic optimization concepts, and motivates the use of optimization models and methods to engineering and scientific practice applications. It establishes key concepts, such as the types of variables, arguments to an optimization problem as continuous, integer and control functions (for optimal control problems). Further, it introduces types of optimization problems according to their formulation (such as multiobjective, bilevel, stochastic optimization problems)

This chapter introduces concepts of norm-1 and infinity norm fitting, both in terms of their own merit as useful fitting techniques, apart from least squares, but also importantly to teach how optimization problems that seem hard to solve (such as by being non-differentiable) can be reformulated effectively into easier ones that can be handled by standard solution methods – in this case by LP solvers.

Unconstrained multivariate gradient-based minimization is introduced by means of search direction-producing methods, focusing on steepest descent and Newton's method. Issues with both methods are discussed, highlighting what happens in the case of locally nonconvex functions, particularly in Newton's method. Linesearch is introduced, effectively rendering multidimensional optimization into a sequence of one-dimensional searches along the ray of the search directions produced. Linesearch criteria are discussed, such as the Armijo first condition, and efficient ways to cut the step size are discussed.

Duality theory has a central role in constrained optimization, both from a theoretical point of view and to enable understanding of solution methods and problem reformulations for special classes of problems. Such applications are presented in the next chapter on Lagrangian relaxation and Lagrangian decomposition. In this chapter, the fundamental background for duality theory is presented along with a basic introduction of key concepts related to it.

Convexity is of paramount importance in optimization theory. This chapter adopts a simple and intuitive description, highlighting the importance of these properties to guarantee global optimality, and paves the way to understanding nonconvex optimization problems in later chapters.

This chapter is the main chapter of the book that introduces in detail how modern Interior Point Methods work, what they are based on, and the associated numerical-computational implementation schemes involved. The difference between primal barrier methods and primal-dual barrier methods is presented and discussed, showing why nowadays mostly primal-dual methods are used in general optimization solvers.

This chapter is a first introduction to penalty and barrier methods, as a direct way to transform generally constrained optimization problems to unconstrained ones. This is done through the appropriate choice of penalty and barrier functions, with the various problems facing such methods highlighted in intuitive and illustrative ways via discussion and graphical examples. The chapter also prepares the reader for the much more advanced material that follows in the next chapter.

This chapter is a standard section in most introductory material on optimization. It examines three basic one-dimensional optimization methods, highlighting connections between them and leading to the one-dimensional Newton's method as the method of choice.

This chapter focuses on the formulation of LP problems as a means to teach how to derive mathematical programming formulations for basic descriptions, specifications, and data related to associated processes that are to be optimized. It focuses on basic blending type problems, production planning, with special focus given to network flow problems that cover a wide range of applications. This topic is revisited in more detail in chapter 14.