Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Home
  • Home
Chapter
  • Get access
    Check if you have access via personal or institutional login
    Log in Register
  • Cited by 1
  • Print publication year: 2011
  • Online publication date: June 2011

14 - Sorting and Fast Algorithms

Export citation

Summary

Introduction

The theme of this chapter is a simple one: there may be better, faster ways of computing something than you may ever have thought of. One of the maxims of computer science is that just a few programs use most of the resources. Early in the history of computing, people recognized that much of the computing resources went into the common task of sorting a list of numbers. If this task could be done faster, then everything could be done better. A concentrated effort on improving sorting algorithms led to several breakthroughs, all following the principle known as “divide and conquer.” In simple terms, to solve a large task, break it into smaller tasks of the same kind. Cleverly done, the resulting algorithm can be more efficient than anyone would have expected, earning the jargon adjective “fast.” The principle of divide and conquer will be discussed in the next section, followed by a discussion of fast algorithms for sorting. Section 14.4 comprises statistical applications of divide and conquer. Another great breakthrough, the fast Fourier transform (FFT), will be discussed in Section 14.5. Using the FFT to compute convolutions will be discussed in Section 14.6, followed by some interesting applications of the FFT to statistics in Section 14.7. This chapter will close with some topics that are important but don't really fit elsewhere: algorithms for constructing permutations and combinations.

Related content

Powered by UNSILO
References
Aho, Alfred V., Hopcroft, John E., and Ullman, Jeffrey D. (1974), The Design and Analysis of Computer Algorithms. Reading, MA: Addison-Wesley.
Baglivo, Jenny, Olivier, Donald, and Pagano, Marcello (1992), “Methods for Exact Goodness-of-Fit Tests,” Journal of the American Statistical Association 87: 464–9.
Bluestein, L. I. (1970), “A Linear Filtering Approach to the Computation of the Discrete Fourier Transform,” IEEE Transactions on Audio and Electroacoustics 18: 451–5.
Bohman, Harald (1975), “Numerical Inversions of Characteristic Functions,” Scandinavian Actuarial Journal, pp. 121–4.
Boothroyd, J. (1967), “Algorithm 29: Permutation of the Elements of a Vector,” Computer Journal 10: 311.
Cooley, J. W., Lewis, P. A., and Welch, P. D. (1967), “Historical Notes on the Fast Fourier Transform,” IEEE Transactions on Audio and Electroacoustics 15: 76–9.
Cooley, J. W. and Tukey, J. W. (1965), “An Algorithm for the Machine Calculation of Complex Fourier Series,” Mathematics of Computation 19: 297–301.
Cox, D. R. and Hinkley, D. V. (1974), Theoretical Statistics. London: Chapman & Hall.
David, F. N. and Barton, D. E. (1962), Combinatorial Chance. New York: Hafner.
Davies, R. B. (1975), “Numerical Inversion of a Characteristic Function,” Biometrika 60: 415–17.
Davies, Robert B. (1980), “ASS 155: The Distribution of a Linear Combination of Chi-Squared Random Variables,” Applied Statistics 29: 323–33.
Good, I. J., Gover, T. N., and Mitchell, G. J. (1970), “Exact Distribution for χ2 and for the Likelihood-Ratio Statistic for the Equiprobable Multinomial Distribution,” Journal of the American Statistical Association 65: 267–83.
Hoare, C. A. R. (1961), “Algorithm 63 (PARTITION) and Algorithm 65 (FIND),” Communications of the ACM 4: 321–2.
Hodges, J. L. and Lehmann, E. L. (1963), “Estimates of Location Based on Rank Tests,” Annals of Mathematical Statistics 34: 598–611.
Imhof, J. P. (1961), “Computing the Distribution of Quadratic Forms in Normal Variables,” Biometrika 48: 419–26.
Johnson, D. B. and Kashdan, S. D. (1978), “Lower Bounds for Selection in X + Y and Other Multisets,” Journal of the ACM 25: 556–70.
Knott, Gary D. (1974), “A Numbering System for Combinations,” Communications of the ACM 17: 45–6.
Knuth, Donald E. (1997), The Art of Computer Programming (vol. 2: Seminumerical Algorithms), 3rd ed. Reading, MA: Addison-Wesley.
Knuth, Donald E. (1998), The Art of Computer Programming (vol. 3: Sorting and Searching), 2nd ed. Reading, MA: Addison-Wesley.
Zeng-Hua, Lu and King, Maxwell L. (2002), “Improving the Numerical Technique for Computing the Accumulated Distribution of a Quadratic Form in Normal Variables,” Econometric Reviews 21: 149–65.
McKean, J. W. and Ryan, T. A. (1977), “Algorithm 516: An Algorithm for Obtaining Confidence Intervals and Point Estimates Based on Rank in a Two-Sample Location Problem,” ACM Transactions on Mathematical Software 3: 183–5.
Mehta, Cyrus R. and Patel, Nitin R. (1986), “FEXACT: A Fortran Subroutine for Fisher's Exact Test on Unordered r × c Contingency Tables,” ACM Transactions on Mathematical Software 12: 154–61.
Monahan, John F. (1984), “Fast Computation of the Hodges–Lehmann Location Estimator,” ACM Transactions on Mathematical Software 10: 265–70.
Monahan, John F. (1987), “An Alternative Method for Computing Overflow Probabilities,” Communications in Statistics A 16: 3355–7.
Nijenhius, A. and Wilf, H. S. (1978), Combinatorial Algorithms. New York: Academic Press.
Pagano, Marcello and Tritchler, David (1983), “On Obtaining Permutation Distributions in Polynomial Time,” Journal of the American Statistical Association 78: 435–40.
Rabiner, L. R., Schafer, R. W., and Rader, C. M. (1969), “The Chirp-z Transform Algorithm and Its Applications,” Bell System Technical Journal 48: 1249–92.
Ramakrishna, M. V. (1987), “Computing the Probability of Hash Table/Urn Overflow,” Communications in Statistics A 16: 3343–53.
Robinson, Ian and Sheather, Simon (1988), “Fast Computation of the Hodges–Lehmann Estimator and Its Associated Confidence Limits,” American Statistical Association Proceedings of the Statistical Computing Section, pp. 187–91.
Shen, Mok-Kong (1963), “Algorithm 202: Generation of Permutations in Lexicographical Order,” Communications of the ACM 6: 517.
Silverman, B. W. (1982), “Kernel Density Estimation Using the Fast Fourier Transform,” Applied Statistics 31: 93–7.
Singleton, R. C. (1969), “An Algorithm for Computing the Mixed-Radix Fast Fourier Transform,” IEEE Transactions on Audio and Electroacoustics 17: 93–103.
Spino, Cathie and Pagano, Marcello (1991), “Efficient Calculation of the Permutation Distribution of Trimmed Means,” Journal of the American Statistical Association 86: 729–37.
Tritchler, David (1984a), “On Inverting Permutation Tests,” Journal of the American Statistical Association 79: 200–7.
Tritchler, David (1984b), “An Algorithm for Exact Logistic Regression,” Journal of the American Statistical Association 79: 709–11.
Wells, Mark B. (1961), “Generation of Permutations by Transposition,” Mathematics of Computation 15: 192–5.