Book contents
- Frontmatter
- Contents
- Preface
- 1 Getting Started
- 2 Perceptron Learning – Basics
- 3 A Choice of Learning Rules
- 4 Augmented Statistical Mechanics Formulation
- 5 Noisy Teachers
- 6 The Storage Problem
- 7 Discontinuous Learning
- 8 Unsupervised Learning
- 9 On-line Learning
- 10 Making Contact with Statistics
- 11 A Bird's Eye View: Multifractals
- 12 Multilayer Networks
- 13 On-line Learning in Multilayer Networks
- 14 What Else?
- Appendices
- Bibliography
- Index
1 - Getting Started
Published online by Cambridge University Press: 05 June 2012
- Frontmatter
- Contents
- Preface
- 1 Getting Started
- 2 Perceptron Learning – Basics
- 3 A Choice of Learning Rules
- 4 Augmented Statistical Mechanics Formulation
- 5 Noisy Teachers
- 6 The Storage Problem
- 7 Discontinuous Learning
- 8 Unsupervised Learning
- 9 On-line Learning
- 10 Making Contact with Statistics
- 11 A Bird's Eye View: Multifractals
- 12 Multilayer Networks
- 13 On-line Learning in Multilayer Networks
- 14 What Else?
- Appendices
- Bibliography
- Index
Summary
In the present chapter we introduce the basic notions necessary to study learning problems within the framework of statistical mechanics. We also demonstrate the efficiency of learning from examples by the numerical analysis of a very simple situation. Generalizing from this example we will formulate the basic setup of a learning problem in statistical mechanics to be discussed in numerous modifications in later chapters.
Artificial neural networks
The statistical mechanics of learning has been developed primarily for networks of so-called formal neurons. The aim of these networks is to model some of the essential information processing abilities of biological neural networks on the basis of artificial systems with a similar architecture. Formal neurons, the microscopic building blocks of these artificial neural networks, were introduced more than 50 years ago by McCulloch and Pitts as extremely simplified models of the biological neuron [1]. They are bistable linear threshold elements which are either active or passive, to be denoted in the following by a binary variable S = ±1. The state Si of a given neuron i changes with time because of the signals it receives through its synaptic couplings Jij from either the “outside world” or other neurons j.
More precisely, neuron i sums up the incoming activity of all the other neurons weighted by the corresponding synaptic coupling strengths to yield the post-synaptic potential ∑jJij Sj and compares the result with a threshold θi specific to neuron i.
- Type
- Chapter
- Information
- Statistical Mechanics of Learning , pp. 1 - 13Publisher: Cambridge University PressPrint publication year: 2001