Dynamic Programming

Multi-stage decision process arises during the study of different mathematical problems; the dynamic programming technique was created for taking care of these sorts of problems. Consider a physical system, at any time t, the state of this system is found out by a set of quantities that is called state variables or state parameters. At certain times, we are call upon to make a decision that will have an effect on the state of the system. This time may be prescribed in advanced or the process itself determines the time. The transformations of the state variables are equivalent to these decisions, the selection of a decision being identical with the choice of a transformation. The future state is guided by the outcome of the preceding decision, which maximizes some function of the parameters that describe the final state.

There are many practical problems where sequential decisions are made at various points in time, at various points in space, and at various levels, say, for a system, for a subsystem, or even for a component level. The problems for which the decisions are required to make sequentially are termed as sequential decision problems. They are also called multistage decision problems because for these problems decisions are to be made at a number of stages. Dynamic programming is a mathematical technique, which is most suitable for the optimization of multistage decision problems. Richard Bellman in the early 1950s [Bellman, (1953)] developed this technique. During application of dynamic programming method, a multi-dimensional decision problem was decomposed into a series of single stage decision problems. In this way, an N-variable problem can be expressed as a series of N single-variable problems. These N number of single-variable problems can be solved sequentially. Most of the time, these single-variable subproblems are quite easier to solve compared to the original problem. This decomposition to N single-variable subproblems is accomplished in such a way that the optimal solution of the original N-dimensional problem can be attained from the optimal solutions of the N one-dimensional problems. It is essential to make a note that the methods used to solve these one-dimensional stages do not affect the overall result. Any method can be used for this purpose; these methods may vary from an easy enumeration method to a relatively complicated technique like differential calculus or a nonlinear programming.