This chapter provides, in the analytic case, a numerical approximation theory of the optimal quadratic cost problem over an infinite time interval whose continuous counterpart was studied in Chapter 2. Assumptions and notation of Chapter 2 are essentially retained. Because of the additional complexity in notation in the discrete case, a glossary of symbols is provided at the end of the chapter. Particular emphasis is placed on approximating the Riccati operator P, the gain operator B* P, the optimal controls u0, the optimal solution y0; etc. This is provided by the main Theorems 4.1.4.1 and 4.1.4.2, which apply to any consistent approximation scheme. They provide, generally, rates of convergence. Under the general setting of Theorems 4.1.4.1 and 4.1.4.2, the rates of convergence are not optimal, however. A more specialized theory, to be presented in Section 4.6 (Theorems 4.6.2.1 and 4.6.2.2), provides optimal rates of convergence. Applications of these results to several examples of partial differential equations of parabolic type with boundary/point control will be given in the subsequent Chapter 5, where numerical schemes will be presented where all the assumptions required by Theorems 4.1.4.1 and 4.1.4.2 will be satisfied. In addition, however, Chapter 5 will provide numerical schemes that will guarantee optimal rates of convergence in the case of the heat equation with Dirichlet or Neumann boundary control according to Theorems 4.6.2.1 and 4.6.2.2.