Book contents
- Frontmatter
- Contents
- Preface
- 1 Getting Started
- 2 Perceptron Learning – Basics
- 3 A Choice of Learning Rules
- 4 Augmented Statistical Mechanics Formulation
- 5 Noisy Teachers
- 6 The Storage Problem
- 7 Discontinuous Learning
- 8 Unsupervised Learning
- 9 On-line Learning
- 10 Making Contact with Statistics
- 11 A Bird's Eye View: Multifractals
- 12 Multilayer Networks
- 13 On-line Learning in Multilayer Networks
- 14 What Else?
- Appendices
- Bibliography
- Index
14 - What Else?
Published online by Cambridge University Press: 05 June 2012
- Frontmatter
- Contents
- Preface
- 1 Getting Started
- 2 Perceptron Learning – Basics
- 3 A Choice of Learning Rules
- 4 Augmented Statistical Mechanics Formulation
- 5 Noisy Teachers
- 6 The Storage Problem
- 7 Discontinuous Learning
- 8 Unsupervised Learning
- 9 On-line Learning
- 10 Making Contact with Statistics
- 11 A Bird's Eye View: Multifractals
- 12 Multilayer Networks
- 13 On-line Learning in Multilayer Networks
- 14 What Else?
- Appendices
- Bibliography
- Index
Summary
In this book we have discussed how various aspects of learning in artificial neural networks may be quantified by using concepts and techniques developed in the statistical mechanics of disordered systems. These methods grew out of the desire to understand some strange low-temperature properties of disordered magnets; nevertheless their usefulness for and efficiency in the analysis of a completely different class of complex systems underlines the generality and strength of the principles of statistical mechanics.
In this final chapter we have collected some additional examples of non-physical complex systems for which an analysis using methods of statistical mechanics similar to those employed for the study of neural networks has given rise to new and interesting results. Compared with the previous chapters, the discussions in the present one will be somewhat more superficial – merely pointing to the qualitative analogies with the problems elucidated previously, rather than working out the consequences in full detail. Moreover, some of the problems we consider are strongly linked to information processing and artificial neural networks, whereas others are not. In all cases quenched random variables are used to represent complicated interactions which are not known in detail, and the typical behaviour in a properly defined thermodynamic limit is of particular interest.
Support vector machines
The main reason which prevents the perceptron from being a serious candidate for the solution of many real-world learning problems is that it can only implement linearly separable Boolean functions.
- Type
- Chapter
- Information
- Statistical Mechanics of Learning , pp. 259 - 274Publisher: Cambridge University PressPrint publication year: 2001