Machine learning and speech and language processing

Speech and language processing is one of the most successful examples of applying machine learning techniques to real problems. Current speech and language techniques embody our real-world information processing, automatically including information extraction, question answering, summarization, dialog, conversational agent, and machine translation (Jurafsky & Martin 2000). Among these, one of the most exciting applications of speech and language processing is speech recognition based voice search technologies (by Google, Nuance) and conversational agent technologies (by Apple) (Schalkwyk, Beeferman, Beaufays et al. 2010). These successful applications started to make people in general casually use speech interface rather than text interface in mobile devices, and the applications of speech and language processing are widely expanding.

One of the core technologies of speech and language processing is automatic speech recognition (ASR) and related techniques. Surprisingly, these techniques are fully based on statistical approaches by using large amounts of data. The machine learning techniques are applied to utilize these data. For example, the main components of ASR are acoustic and language models. The acoustic model (AM) provides a statistical model of each phoneme/word unit, and it is represented by a hidden Markov model (HMM). The HMM is one of the most typical examples of dealing with sequential data based on machine learning techniques (Bishop 2006), and machine learning techniques provide an efficient method of computing a maximum likelihood value for the HMM and an efficient training algorithm of the HMM parameters. The language model (LM) also provides an n-gram based statistical model for word sequences, which is also trained by using the large amount of data based on machine learning techniques. These statistical models and their variants are used for the other speech and language applications, including speaker verification and information retrieval, and thus, machine learning is a core component of speech and language processing.

Machine learning covers a wide range of applications in addition to speech and language processing, including bioinformatics, data mining, and computer vision. Machine learning also covers various theoretical fields including pattern recognition, information theory, statistics, control theory, and applied mathematics. Therefore, many people are studying and developing machine learning techniques, and the progress of machine learning is rather fast.