Hostname: page-component-84b7d79bbc-dwq4g Total loading time: 0 Render date: 2024-07-27T17:41:26.162Z Has data issue: false hasContentIssue false
  • English
  • Français

Selecting the architecture of a class of back-propagation neural networks used as approximators

Published online by Cambridge University Press:  27 February 2009

William C. Carpenter  and
Margery E. Hoffman
William C. Carpenter
Affiliation:
Department of Civil and Environmental Engineering, University of South Florida, Tampa, FL 33620, USA
Margery E. Hoffman
Affiliation:
Naval Air Warfare Center, Aircraft Division, Patuxent River, MD 20670, USA
Get access Rights & Permissions [Opens in a new window]

Abstract

This paper examines the architecture of back-propagation neural networks used as approximators by addressing the interrelationship between the number of training pairs and the number of input, output, and hidden layer nodes required for a good approximation. It concentrates on nets with an input layer, one hidden layer, and one output layer. It shows that many of the currently proposed schemes for selecting network architecture for such nets are deficient. It demonstrates in numerous examples that overdetermined neural networks tend to give good approximations over a region of interest, while underdetermined networks give approximations which can satisfy the training pairs but may give poor approximations over that region of interest. A scheme is presented that adjusts the number of hidden layer nodes in a neural network so as to give an overdetermined approximation. The advantages and disadvantages of using multiple output nodes are discussed. Guidelines for selecting the number of output nodes are presented.

Keywords

Neural NetworksAppoximationsResponse SurfacesMemorizationGeneralization
Type
Articles
Information
AI EDAM , Volume 11 , Issue 1 , January 1997 , pp. 33 - 44
Copyright
Copyright © Cambridge University Press 1997

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

REFERENCES

Amirikian, B., & Nishimura, H. (1994). What size network is good for generalization of a specific task of interest? Neural Networks 7(2), 321–329.CrossRefGoogle Scholar
Anderson, D., Hines, E.L., Authur, S.J., & Eiap, E.L. (1993). Application of artificial neural networks to prediction of minor axis steel connections. In Neural Networks and Combinatorial Optimization in Civil and Structural Engineering. Civil-Comp Press Limited, Edinburgh, UK.Google Scholar
Anderson, J., & Rosenfeld, E. (1988). Neurocomputing: Foundations of Research. MIT Press, Cambridge, MA.CrossRefGoogle Scholar
Bailey, D., & Thompson, D. (1990). How to develop neural network applications. AI Expert, 5(6), 38–47.Google Scholar
Baum, E.F., & Haussler, D. (1988). Neural Computation 1. MIT Press, Cambridge, MA, pp. 151–160.Google Scholar
Berke, L., & Hajela, P. (1992). Application of artificial neural nets in structural mechanics. In Shape and Layout Optimization of Structural Systems (Rozvany, G.I.N., Ed.) (CISM lecture series, Udine, Italy, 1990). Springer-Verlag, Vienna.Google Scholar
Bose, N.K., & Liang, P. (1996). Neural Network Fundamentals with Graphs, Algorithms, and Applications. McGraw-Hill, Inc., New York, p. 242.Google Scholar
Box, G.E.P., & Draper, N.R. (1987). Empirical Model-Building and Response Surfaces. John Wiley and Sons, New York.Google Scholar
Carpenter, W.C. (1993). Effect of design selection on response surface performance. Final Report, NASA Grant Number NAG–1–1378.Google Scholar
Carpenter, W.C., & Barthelemy, J.-F.M. (1993). A comparison of polynomial approximations and artificial neural nets as response surfaces. Struct. Opt. 5, 1–15.CrossRefGoogle Scholar
Carpenter, W.C., & Barthelemy, J.-F.M. (1994). Common misconceptions about neural networks as approximators. J. Compul. Civ. Engrg. 8(3), 345–358.CrossRefGoogle Scholar
Carpenter, W.C., & Hoffman, M.E. (1995). Training backprop neural networks. Al Expert 10(3), March, 30–33.Google Scholar
Caudill, M. (1991). Neural network training tips and techniques. Al Expert, 6(1), 56–61.Google Scholar
Demuth, H., & Beale, M. (1992). Neural Network Toolbox, v. 1.0 User’s Guide. The MathWorks, Inc., Gochituate Place, 24 Prime Park Way, Natick, MA 01760.Google Scholar
Fujita, O. (1992). Optimization of the hidden unit function in feedforward neural networks. Neural Networks 5, 755–764.CrossRefGoogle Scholar
Greville, T.N.E. (1959). The pseudoinverse of a rectangular or singular matrix and its application to the solution of systems of linear equations. S1AM Rev. 1(1), 38–43.Google Scholar
Hajela, P., & Berke, L. (1990). Neurobiological computational models in structural analysis and design. Paper AIAA–90–1133–CP, AIAA/ASME/ASCE/AHS/ASC 31st Structures, Structural Dynamics and Materials Conference, New York, NY.Google Scholar
Hoffman, M. (1993). An investigation of artificial neural networks for F-14A wing-sweep angle calculations. Report No. NAWCADWAR-93028–6. Naval Air Warfare Center, Aircraft Division, Warminster, PA.Google Scholar
Hornik, K., Stinchcombe, M., & White, H. (1989). Multilayer feedforward networks are universal approximators. Neural Networks 2, 359–366.CrossRefGoogle Scholar
Kahkonen, K., & Pallas, J. (1993). Roles, benefits and objectives of neural networks in building construction practice. In Neural Networks and Combinatorial Optimization in Civil and Structural Engineering. Civil-Comp Press Limited, Edinburgh, UK.Google Scholar
Khuri, A.I., & Cornell, J.A. (1987). Response Surfaces, Designs and Analyses. Marcel Dekker Inc, New York.Google Scholar
Kirkegaard, P.H., & Rytter, A. (1993). The use of neural networks for damage detection and location in a steel member. In Neural Networks and Combinatorial Optimization in Civil and Structural Engineering. Civil-Comp Press Limited, Edinburgh, UK. (Vol. 6, No. 11).Google Scholar
Lawrence, J. (1991). Data preparation for a neural network. Al Expert, 34–40.Google Scholar
Liong, S.Y., & Chan, W.T. (1993). Runoff volume estimates with neural networks. In Neural Networks and Combinatorial Optimization in Civil and Structural Engineering. Civil-Comp Press Limited, Edinburgh, UK.Google Scholar
Lippmann, R.P. (1987). An introduction to computing with neural nets. IEEE ASSP Mag. 4(2), 4–22.CrossRefGoogle Scholar
Lu, P.-C, & Urquidi-Macdonald, M. (1994). Prediction of IGSCC in TYPE 304 SS using an artificial neural network. Paper 151 Corrosion94, The Annual Conference and Corrosion Show, sponsored by NACE International, Houston, TX, 151/1–151/20.Google Scholar
Lyons, G., & Hunt, J. (1993). Traffic models—A role for neural networks? In Neural Networks and Combinatorial Optimization in Civil and Structural Engineering. Civil-Comp Press Limited, Edinburgh, UK.Google Scholar
Mawdesley, M.J., & Carr, V. (1993). Artificial neural networks for construction project planning. In Neural Networks and Combinatorial Optimization in Civil and Structural Engineering. Civil-Comp Press Limited, Edinburgh, UK.Google Scholar
Moody, J. (1992). The effective number of parameters: An analysis of generalization and regularization in nonlinear learning systems. In Advances in Neural Information Processing 4 (Moody, J.E., Hanson, S.F., and Lippmann, R.P., Eds.), pp. 847–854. Morgan Kaufmann, Denver, CO.Google Scholar
Myers, R.H. (1971). Response Surface Methodology. Allyn and Bacon, Boston, MA.Google Scholar
Penrose, R. (1955). A generalized inverse for matrices. Proc. Cambridge Phil. Soc. 51, 406–413.CrossRefGoogle Scholar
Rogers, K.L. (1994). Simulating structural analysis with neural nets. J. Comput. Civ. Engrg. 8(2), 252–265.CrossRefGoogle Scholar
Rumelhart, D., & McClelland, J. (1986). Parallel Distributed Processing, Vol I and II. MIT Press, Cambridge, MA.CrossRefGoogle Scholar
Satori, M.A., & Antsaklis, P.J. (1991). A simple method to derive bounds on the size and to train multilayer neural networks. IEEE Trans. Neural Networks 2(4), 467–471.CrossRefGoogle Scholar
Swift, R.A., & Batill, S.M. (1991). Application of neural networks to preliminary structural design. AIAA/ASME/AHS/ASC 32nd Structures, Structural Dynamics and Materials Conference, Baltimore, MD, April 8–10, pp. 335–343.Google Scholar
Taha, O., & Ghosh, J. (1995). Controlling water reservoirs using a hybrid intelligent architecture. In Intelligent Engineering Systems Through Artificial Neural Networks, Vol 5. Fuzzy Logic and Evolutionary Programming (Dagli, C.H. et al. , Eds.). ASME Press, New York.Google Scholar
Wang, Z., Di Massimo, C., Tham, M.T., & Morris, A.T. (1994). A procedure for determining the topology of multilayer feedforward neural networks. Neural Networks 7(2), 291–300.CrossRefGoogle Scholar
Weigend, A.S., & Rumelhart, D.E. (1991). The effective dimension of the space of hidden units. In Proc. IEEE Intl. Joint Conf. Neural Networks, Singapore, Nov. 18–21, Vol. 3, pp. 2069–2074.Google Scholar
Williams, T.P. (1993). Neural networks to predict construction cost indexes. In Neural Networks and Combinatorial Optimization in Civil and Structural Engineering. Civil-Comp Press Limited, Edinburgh, UK.Google Scholar
Wu, X., & Lim, S.Y. (1993). Prediction of maximum scour depth at spur dikes with adaptive neural networks. In Neural Networks and Combinatorial Optimization in Civil and Structural Engineering. Civil-Comp Press Limited, Edinburgh, UK.Google Scholar