Introduction

In Chapter 1 we briefly introduced an information channel as a model of a communication link or a related system where the input is a message and the output is an imperfect reproduction of it. In particular we also use this concept as a model of a storage system where input and output are separated in time rather than in space. In our presentation we do not refer to the underlying physical medium or discuss whether it is fundamentally continuous or quantized. The process of transmitting and receiving (writing and reading) is assumed to use finite alphabets, which may well be different, and it is understood that these alphabets represent a digital implementation of processes that make efficient use of the physical medium under the current technological and economic constraints. In this chapter we introduce the fundamentally important concept of channel capacity. It is defined in a straightforward way as the maximum of mutual information; however, the significance becomes clear only as we show how this is actually the amount of information that can be reliably transmitted through the channel. Reliable communication at rates approaching capacity requires the use of coding. For this reason we have chosen to present the basic concepts of channel coding in the same chapter and to emphasize the relation between codes and the information-theoretic quantities. In reality the codes that are used are matched to a few special channels, and other real channels are converted to or approximated by one of these types.