Artificial neural networks (ANN) is a field of research aimed at using complex systems, made of simple identical non-linear parallel elements, for performing different types of tasks; for review see (Hertz et al 1990),(Bishop 1995) and (Ripley 1996). During the years neural networks have been successfully applied to perform regression, classification, control and prediction tasks in a variety of scenarios and architectures. The most popular and useful of ANN architectures is that of layered feed-for ward neural networks, in which the non-linear elements (neurons) are arranged in successive layers, and the information flows unidirectionally; this is in contrast to the other main generic architecture of recurrent networks where feed-back connections are also permitted. Layered networks with an arbitrary number of hidden units have been shown to be universal approximators (Cybenko 1989; Hornik et al 1989) for continuous maps and can therefore be used to implement any function defined in these terms.

Learning in layered neural networks refers to the modification of internal network parameters, so as to bring the map implemented by the network as close as possible to a desired map. Learning may be viewed as an optimization of the parameter set with respect to a set of training examples instancing the underlying rule. Two main training paradigms have emerged: batch learning, in which optimization is carried out with respect to the entire training set simultaneously, and on-line learning, where network parameters are updated after the presentation of each training example (which may be sampled with or without repetition).