Book contents
- Frontmatter
- Contents
- Contributors
- Introduction: Modelling perception with artificial neural networks
- Part I General themes
- Part II The use of artificial neural networks to elucidate the nature of perceptual processes in animals
- Part III Artificial neural networks as models of perceptual processing in ecology and evolutionary biology
- Part IV Methodological issues in the use of simple feedforward networks
- 14 How training and testing histories affect generalisation: a test of simple neural networks
- 15 The need for stochastic replication of ecological neural networks
- 16 Methodological issues in modelling ecological learning with neural networks
- 17 Neural network evolution and artificial life research
- 18 Current velocity shapes the functional connectivity of benthiscapes to stream insect movement
- 19 A model biological neural network: the cephalopod vestibular system
- Index
- References
17 - Neural network evolution and artificial life research
from Part IV - Methodological issues in the use of simple feedforward networks
Published online by Cambridge University Press: 05 July 2011
Book contents
- Frontmatter
- Contents
- Contributors
- Introduction: Modelling perception with artificial neural networks
- Part I General themes
- Part II The use of artificial neural networks to elucidate the nature of perceptual processes in animals
- Part III Artificial neural networks as models of perceptual processing in ecology and evolutionary biology
- Part IV Methodological issues in the use of simple feedforward networks
- 14 How training and testing histories affect generalisation: a test of simple neural networks
- 15 The need for stochastic replication of ecological neural networks
- 16 Methodological issues in modelling ecological learning with neural networks
- 17 Neural network evolution and artificial life research
- 18 Current velocity shapes the functional connectivity of benthiscapes to stream insect movement
- 19 A model biological neural network: the cephalopod vestibular system
- Index
- References
Summary
17.1 Introduction
Neural networks have been employed as research tools both for machine learning applications and the simulation of artificial organisms. In recent times, much research has been undertaken on the evolution of neural networks where the architecture, weights or both are allowed to be determined by an evolutionary process such as a genetic algorithm. Much of this research is carried out with the machine learning and evolutionary computation community in mind rather than the artificial life community and as such, the latter has been slow to adopt innovative techniques which could lead to the development of complex, adaptive neural networks and in addition, shorten experiment development and design times for researchers.
This chapter attempts to address this issue by reminding researchers of the wealth of techniques that have been made available for evolutionary neural network research. Many of these techniques have been refined into freely available and well-maintained code libraries which can easily be incorporated into artificial life projects hoping to evolve neural network controllers.
The first section of this chapter outlines a review of the techniques employed to evolve neural network architectures, weights or both architectures and weights simultaneously. The encoding schemes presented in this chapter describe the encoding of multi-layer feedforward and recurrent neural networks but there are some encoding schemes which can (and have been) employed to generate more complex neural networks such as spiking (Floreano & Mattiussi, 2001; Di Paulo, 2002) and gasNets (Smith et al., 2002) which are beyond the scope of this chapter.
- Type
- Chapter
- Information
- Modelling Perception with Artificial Neural Networks , pp. 334 - 350Publisher: Cambridge University PressPrint publication year: 2010
References
, & 1994. An evolutionary algorithm that constructs recurrent neural networks. IEEE Trans Neural Netw 5(1), 54–65.CrossRefGoogle ScholarPubMed
, & 1992. Evolving networks: using the genetic algorithm with connectionist learning. In Artificial Life II (ed. , , & ), pp. 511–547. Addison-Wesley.Google Scholar
& 1992. Biological Metaphors and the Design of Artificial Neural Networks. Master's thesis, Leiden.
& 2003. Enhancing autonomous agents evolution with learning by imitation. Interdisciplin J Artific Intell Simul Behav 1(4), 335 – 348.Google Scholar
1995. Evolutionary algorithms for neural network design and training. Technical Report No. 322. University of Karlsruhe, Institute AIFB.Google Scholar
, & 1994. Cell division and migration in a ‘genotype’ for neural networks. Network 5, 497–515.CrossRefGoogle Scholar
& 1999. Evolving neural networks to play checkers without relying on expert knowledge. IEEE Trans Neur Netw 10, 1382–1391.CrossRefGoogle ScholarPubMed
& 2006. Increasing population diversity through cultural learning. Adapt Behav 14(4), 315–338.CrossRefGoogle Scholar
1990. Genetic programming: building artificial nervous systems using genetically programmed neural network modules. In Machine Learning: Proceedings of the Seventh International Conference (ed. & ), pp. 132–139. Morgan Kaufmann.Google Scholar
2002. Evolving spike-timing dependent synaptic plasticity for robot control. In EPSRC/BBSRC International Workshop: Biologically-inspired Robotics, The Legacy of W. Grey Walter, WGW2002.
& 2001. Evolution of spiking neural controllers for autonomous vision-based robots. In Evolutionary Robotics. From Intelligent Robotics to Artificial Life (ed. T. Gomi), pp. 38–61. Springer.Google Scholar
& 2008. On the coupling between agent internal and agent/ environmental dynamics: development of spatial representations in evolving autonomous robots. Adaptive Behav 16(2–3), 148–165.CrossRefGoogle Scholar
1989. Genetic Algorithms in Search, Optimization and Machine Learning. Addison-Wesley.Google Scholar
1994. Neural Network Synthesis using Cellular Encoding and the Genetic Algorithm. PhD thesis, Centre d'etude nucleaire de Grenoble, Ecole Normale Superieure de Lyon.
1995. Automatic definition of modular neural networks. Adapt Behav 3(2), 151–183.CrossRefGoogle Scholar
, & 1996. A comparison between cellular encoding and direct encoding for genetic neural networks. In Genetic Programming 1996: Proceedings of the First Annual Conference, pp. 81–89.
, , & . 2007. Evolving a neural model of insect path integration. Adapt Behav 15(3), 273–287.CrossRefGoogle Scholar
& . 1991. Genetic synthesis of neural network architecture. In Handbook of Genetic Algorithms (ed. L. Davis), pp. 202–221. Van Nostrand Reinhold.Google Scholar
, , & . 1996. Using the genetic algorithm to build optimal neural networks for fault-prone module detection. In Proceedings of the Seventh International Symposium on Software Reliability Engineering, pp. 152–162. IEEECS.Google Scholar
. 1975. Adaptation in Natural and Artificial Systems. The University of Michigan Press.Google Scholar
& . 1998. Genetic encoding of neural networks using attribute grammars. In CITO Researcher Retreat, Hamilton, Ontario, Canada.Google Scholar
. 1990. Designing neural networks using genetic algorithm with graph generation system. Complex Syst 4, 461–476.Google Scholar
. 1994. Combining genetic algorithms and neural networks: The encoding problem. Master's thesis, University of Erlangen and The University of Tennessee.
. 1996. Genetic encoding strategies for neural networks. In Proceedings of Information Processing and Management of Uncertainty in Knowledge-Based Systems.
& . 1991. Back propagation is sensitive to initial conditions. Adv Neural Inform Process Syst 3, 860 – 867.Google Scholar
& . 1991. Genetic generation of both the weights and architecture for a neural network. In International Joint Conference on Neural Networks, IJCNN-91, Vol. II, pp. 397–404. IEEE Computer Society Press.Google Scholar
& . 1996. Evolving graphs and networks with edge encoding: preliminary report. In Late Breaking Papers at the Genetic Programming 1996 Conference Stanford University (ed. J. R. Koza), pp. 117–124. Stanford University.Google Scholar
. 1993. Representation and evolution of neural networks. In Artificial Neural Nets and Genetic Algorithms. Proceedings of the International Conference at Innsbruck, Austria (ed. , & ), pp. 643–649. Springer.Google Scholar
1993. Searching among search spaces: Hastening the genetic evolution of feedforward neural networks. In Artificial Neural Nets and Genetic Algorithms. Proceedings of the International Conference at Innsbruck, Austria (ed. , & ), pp. 635–643. Springer-Verlag.Google Scholar
, & . 1989. Designing neural networks using genetic algorithms. In Proceedings of the Third International Conference on Genetic Algorithms and Their Applications, pp. 379–384.
& . 1989. Training feedforward neural networks using genetic algorithms. In Proceedings of the Eleventh International Joint Conference on Artificial Intelligence, pp. 762–767. Morgan Kaufmann.Google Scholar
& . 1995. Discovering complex othello strategies through evolutionary neural networks. Connect Sci 7(3–4), 195–209.Google Scholar
& . 1992. Growing neural networks. Technical Report. Institute of Psychology, CNR Rome.Google Scholar
, , & 1997. Evolving neural networks to play Go. In Proceedings of the 7th International Conference on Genetic Algorithms, East Lansing, MI.Google Scholar
& . 1999. Evolving learnable neural networks under changing environments with various rates of inheritance of acquired characters: comparison between Darwinian and Lamarckian evolution. Artific Life 5(3), 203–223.CrossRefGoogle ScholarPubMed
, & 1993. Application of genetic algorithms to the construction of topologies for multilayer perceptrons. In Proceedings of the International Conference on Artificial Neural Networks and Genetic Algorithms, pp. 675–682.
& . 1998. A comparison of matrix rewriting versus direct encoding for evolving neural networks. In Proceedings of the 1998 IEEE International Conference on Evolutionary Computation, pp. 392 – 397.
, , '. 2002. Neuronal plasticity and temporal adaptivity: Gasnet robot control networks. Adapt Behav 10(3–4), 161–184.CrossRefGoogle Scholar
& 2002. Efficient evolution of neural network topologies. In Proceedings of the 2002 Congress on Evolutionary Computation CEC 2002 (ed. et al.), pp. 1757–1762. IEEE Press.Google Scholar
& 2002. Evolving neural networks through augmenting topologies. Evol Comput 10(2), 99–127.CrossRefGoogle ScholarPubMed
. 1986. Two problems with backpropagation and other steepest-descent learning procedures for networks. In Proceedings of 8th Annual Conference of the Cognitive Science Society, pp. 823–831.
. 1994. GANNet: A Genetic Algorithm for Searching Topology and Weight Spaces in Neural Network Design. PhD thesis, University of Maryland College Park.Google Scholar
, & . 1990. Genetic algorithms and neural networks – optimizing connections and connectivity. Parallel Comput 14(3), 347–361.CrossRefGoogle Scholar
. 1999. Evolving artificial neural networks. In Proceedings of the IEEE, pp. 1423–1447.
& . 1996. Evolving artificial neural networks through evolutionary programming. In Proceedings of the 5 th Annual Conference on Evolutionary Programming, pp. 257–266. MIT Press.Google Scholar
& . 1993. Evolving optimal neural networks using genetic algorithms with Occam's razor. Complex Syst 7(3), 199–220.Google Scholar