In general, speech and language processing involves extensive knowledge of statistical models. The acoustic model using hidden Markov models and the language model using n–grams are mainly introduced here. Both acoustic and language models are important parts of modern speech recognition systems where the learned models from real-world data are full of complexity, ambiguity, and uncertainty. The uncertainty modeling is crucial to tackle the lack of robustness for speech and language processing.

This book addresses fundamental theories of Bayesian learning, inference, and prediction for the uncertainty modeling. Uniquely, compared with standard textbooks for dealing with the fundamental Bayesian approaches, this book focuses on the practical methods of the approaches to make them applicable to actual speech and language problems. We (the authors) have been studying these topics for a long time with a strong belief that the Bayesian approaches could solve “robustness” issues in speech and language processing, which are the most difficult problem and most serious shortcoming of real systems based on speech and language processing. In our experience, the most difficult issue in applying Bayesian approaches is how to appropriately choose a specific technique among the many Bayesian techniques proposed in statistics and machine learning so far. One of our answers to this question is to provide the approximated Bayesian inference methods rather than focusing on covering the whole Bayesian techniques. We categorize the Bayesian approaches into five categories: the maximum a-posteriori estimation; evidence approximation; asymptotic approximation; variational Bayes; and Markov chain Monte Carlo. We also describe the speech and language processing applications within this categorization so that readers can appropriately choose the approximated Bayesian techniques for their problems.

This book is part of our long-term cooperative efforts to promote the Bayesian approaches in speech and language processing. We have been pursuing this goal for more than ten years, and part of our efforts was to organize a tutorial lecture with this theme at the 37th International Conference on Acoustics, Speech, and Signal Processing (ICASSP) in Kyoto, Japan, March 2012. The success of this tutorial lecture prompted the idea of writing a textbook with this theme. We strongly believe in the importance of the Bayesian approaches, and we sincerely encourage the researchers who work with Bayesian speech and language processing.