This chapter discusses a generic least squares method and a special situation when the base functions are orthogonal to each other, which makes the solution explicit; in addition, we learn that the essence of the least squares method can be viewed as a way to project the target function in a higher dimension onto a lower dimension formed by the base functions. The least squares method ensures that the error vector is “perpendicular” to the projected (or approximate) vector in the base function dimension (a lower dimension) and thus has the shortest “length” or minimized error. Although this chapter does not have much computation involved, it is very important for a good understanding of the meaning of many techniques and methods in the subsequent chapters.