Hostname: page-component-76fb5796d-wq484 Total loading time: 0 Render date: 2024-04-26T09:53:20.413Z Has data issue: false hasContentIssue false
  • English
  • Français

Gradient algorithms for principal component analysis

Published online by Cambridge University Press:  17 February 2009

R. E. Mahony ,
U. Helmke  and
J. B. Moore
R. E. Mahony
Affiliation:
Dept of Systems Eng, Research School of Phys. Sciences and Eng, ANU, Canberra ACT 0200
U. Helmke
Affiliation:
Dept of Mathematics, University of Regensburg, 8400 Regensburg, F.R.G.
J. B. Moore
Affiliation:
Dept of Systems Eng, Research School of Phys. Sciences and Eng, ANU, Canberra ACT 0200
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

The problem of principal component analysis of a symmetric matrix (finding a p-dimensional eigenspace associated with the largest p eigenvalues) can be viewed as a smooth optimization problem on a homogeneous space. A solution in terms of the limiting value of a continuous-time dynamical system is presented. A discretization of the dynamical system is proposed that exploits the geometry of the homogeneous space. The relationship between the proposed algorithm and classical methods are investigated.

Type
Research Article
Information
The ANZIAM Journal , Volume 37 , Issue 4 , April 1996 , pp. 430 - 450
Copyright
Copyright © Australian Mathematical Society 1996

References

[1]Ammar, G. and Martin, C., “The geometry of matrix eigenvalue methods”, Acta Applicandae Mathematicae 5 (1986) 239–278.CrossRefGoogle Scholar
[2]Bloch, A. M., “A completely integrable Hamiltonian system associated with line fitting in complex vector spaces”, Bull. Amer. Math. Soc. 12 (1985) 250–254.CrossRefGoogle Scholar
[3]Brockett, R. W., Differential geometry and the design of gradient algorithms, Vol. 54 Proc. Symp. Pure Mathematics (1993) 69–92.CrossRefGoogle Scholar
[4]Byrnes, C. I. and Willems, J. C., “Least-squares estimation, linear programming and momentum: A geometric parametrization of local minima”, IMA J. Math. Control and Information 3 (1986) 103–118.CrossRefGoogle Scholar
[5]Chu, M. T., “On the continuous realization of iterative processes”, SIAM Review 30 (1988) 375–387.CrossRefGoogle Scholar
[6]Deift, P., Nanda, T. and Tomei, C., “Ordinary differential equations for the symmetric eigenvalue problem”, SIAM J. ofNumer. Anal. 20 (1983) 1–22.CrossRefGoogle Scholar
[7]Duistermaat, J. J., Kolk, J. A. C. and Varadarajan, V. S., “Functions, flows and oscillatory integrals on flag manifolds and conjugacy classes in real semi-simple Lie groups”, Composito Mathematica 49 (1983) 309–398.Google Scholar
[8]Faddeev, D.K. and Faddeeva, V. N., Computational methods of linear algebra (W. H. Freeman and Co., San Francisco, 1963).Google Scholar
[9]Gibson, C. G., Singular points of smooth mappings, Vol. 25 Res. Notes in Math. (Pitman, London, United Kingdom, 1979).Google Scholar
[10]Golub, G. H. and Loan, C. F. Van, Matrix computations (The Johns Hopkins University Press, Baltimore, Maryland U.S.A., 1989).Google Scholar
[11]Helmke, U. and Moore, J. B., Optimization and dynamical systems, Communications and Control Engineering (Springer-Verlag, London, 1994).CrossRefGoogle Scholar
[12]Horn, R. A. and Johnson, C. R., Matrix analysis (Cambridge University Press, Cambridge, U.K., 1985).CrossRefGoogle Scholar
[13]Mahony, R. E., “Optimization algorithms on homogeneous spaces: with aplications in linear systems theory”, Ph. D. Thesis, Department of Systems Engineering, Canberra, Australia, 1994.Google Scholar
[14]Moore, J. B., Mahony, R. E. and Helmke, U., “Numerical gradient algorithms for eigenvalue and singular value calculations”, SIAM J. Matrix Analysis (1994).CrossRefGoogle Scholar
[15]Oja, E., “A simplified neuron model as a principal component analyzer”, J. Math. Biology 15 (1982) 267–273.CrossRefGoogle ScholarPubMed
[16]Oja, E., “Neural networks, principal components, and subspaces”, Intern. J. Neural Systems 1. (1989) 61–68.CrossRefGoogle Scholar
[17]Warner, F. W., Foundations of differentiable manifolds and Lie groups, Graduate texts in Mathem- atics (Springer-Verlag, New York, U.S.A., 1983).CrossRefGoogle Scholar
[18]Yan, W.-Y., Helmke, U. and Moore, J. B., “Global analysis of Oja's flow for neural networks”, IEEE Tran. Neural Networks 5 (1994), 674–683.Google ScholarPubMed