Skip to main content Accessibility help
Hostname: page-component-568f69f84b-cgcw8 Total loading time: 0.26 Render date: 2021-09-21T21:31:36.513Z Has data issue: true Feature Flags: { "shouldUseShareProductTool": true, "shouldUseHypothesis": true, "isUnsiloEnabled": true, "metricsAbstractViews": false, "figures": true, "newCiteModal": false, "newCitedByModal": true, "newEcommerce": true, "newUsageEvents": true }

Diversity Partitioning Using Shannon's Entropy and its Relationship to Rarefaction

Published online by Cambridge University Press:  21 July 2017

Thomas D. Olszewski*
Department of Geology and Geophysics, MS 3115, Texas A&M University, College Station, Texas 77843-3115
Get access


Diversity (the variety of different types of organisms) of an ecological or paleoecological system reflects processes and history operating across a range of hierarchically related scales. For example, the diversity of a biofacies is the sum of the diversity in all the local patches composing the biofacies, the diversity of a depositional system is composed of all the biofacies composing the depositional system, and the diversity of a biotic province is composed of all the landscapes composing the province. Diversity at a larger scale (γ-diversity) incorporates both the average inventory diversity of units of the next smaller scale (α-diversity) and the compositional differences, or differentiation diversity, among the smaller units (β-diversity). Many familiar means of measuring diversity can be mathematically partitioned to determine the relative contribution of different diversity components at any hierarchical level. When using richness (the number of taxa in an ecological system) as a measurement of diversity, it is necessary to use rarefaction to correct for differences in sample size. The divergence between sample-based and individual-based rarefaction curves of a composite collection (γ-diversity) incorporating all the samples (α-diversity) contributing to a given hierarchical level reflects the degree of non-random compositional difference among the smaller scale units (β-diversity). Alternatively, Shannon's entropy can be partitioned additively: β-entropy equals γ-entropy (based on a composite sample) minus average α-entropy of the constituent samples. A useful property of entropy is that it can be converted to effective richness, the number of taxa that would result in the same entropy value if all were equally abundant. Effective richness can be thought of as a unit conversion from non-intuitive entropy units to more easily understood richness units. Effective richness derived from Shannon's entropy partitions diversity multiplicatively – i.e., β-diversity is the number of compositionally distinct smaller units that contribute to the total diversity at the higher level. Diversity partitioning is rapidly becoming adopted as a tool for directly addressing how the structure of higher-level ecological and paleoecological systems reflects interactions among lower-level units in response to environmental and evolutionary changes.

Ecological Data
Copyright © 2010 by the Paleontological Society 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)


Allan, J. D. 1975. Components of diversity. Oecologia, 18:359367.CrossRefGoogle ScholarPubMed
Anderson, M. J., Ellingsen, K. E., and McArdle, B. H. 2006. Multivariate dispersion as a measure of beta diversity. Ecology Letters, 9:683693.CrossRefGoogle ScholarPubMed
Basharin, G. P. 1959. On a statistical estimate for the entropy of a sequence of independent random variables. Theory of Probability and Its Applications, 29:333336.CrossRefGoogle Scholar
Ben-Naim, A. 2008. Entropy Demystified: The Second Law Reduced to Plain Common Sense with Seven Simulated Games. World Scientific, Hackensack, New Jersey, 223 p.CrossRefGoogle Scholar
Brewer, A. and Williamson, M. 1994. A new relationship for rarefaction. Biodiversity Conservation, 3:373379.CrossRefGoogle Scholar
Bunge, J. and Fitzpatrick, M. 1993. Estimating the number of species: a review. Journal of the American Statistical Association, 88:364373.Google Scholar
Buzas, M. A. and Hayek, L. C. 2005. On richness and evenness within and between communities. Paleobiology, 31:199220.CrossRefGoogle Scholar
Chao, A. and Shen, T.-J. 2003. Nonparametric estimation of Shannon's index of diversity when there are unseen species in sample. Environmental and Ecological Statistics, 10:429443.CrossRefGoogle Scholar
Chiarucci, A., Bacaro, G., Rocchini, D., and Fattorini, L. 2008. Discovering and rediscovering the sample-based rarefaction formula in the ecological literature. Community Ecology, 9:121123.CrossRefGoogle Scholar
Coleman, B. D. 1981. On random placement and species-area relations. Mathematical Biosciences, 54:191215.CrossRefGoogle Scholar
Coleman, B. D., Mares, M. A., Willig, M. R., and Hsiey, Y. 1982. Randomness, area, and species richness. Ecology, 63:11211133.CrossRefGoogle Scholar
Colwell, R. K. and Coddington, J. A. 1994. Estimating terrestrial biodiversity through extrapolation. Philosophical Transactions of the Royal Society of London B, 345:101118.CrossRefGoogle ScholarPubMed
Colwell, R. K., Mao, C. X., and Chang, J. 2004. Interpolating, extrapolating, and comparing incidence-based species accumulation curves. Ecology, 85:27172727.CrossRefGoogle Scholar
Cooper, G. A. and Grant, R. E. 1976. Permian Brachiopods from West Texas, IV. Smithsonian Contributions to Paleobiology, 21:19232607.Google Scholar
Couteron, P. 2004. Additive apportioning of species diversity: towards more sophisticated models and analyses. Oikos, 107:215221.CrossRefGoogle Scholar
Crist, T. O., Veech, J. A., Gering, J. C., and Summerville, K. S. 2003. Partitioning species diversity across landscapes and regions: a hierarchical analysis of α, β, and γ diversity The American Naturalist, 162:734743.CrossRefGoogle ScholarPubMed
Fall, L. M. 2010. Processes Influencing the Diversity of Middle Permian Brachiopods in the Bell Canyon Formation of the Delaware Basin (West Texas, Guadalupe Mountains National Park. Phd Dissertation, Texas A&M University, 191 p.Google Scholar
Gering, J. C., Crist, R. O., and Veech, J. A. 2003. Additive partitioning of species diversity across multiple spatial scales: implications for regional conservation of biodiversity. Conservation Biology, 17:488499.CrossRefGoogle Scholar
Gotelli, N. J. and Colwell, R. K. 2001. Quantifying biodiversity: procedures and pitfalls in the measurement and comparison of species richness. Ecology Letters, 4:379391.CrossRefGoogle Scholar
Hausser, J. and Strimmer, K. 2009. Entropy inference and the James-Stein estimator, with application to nonlinear gene association networks. Journal of Machine Learning Research, 10:14691484.Google Scholar
Hausser, J. and Strimmer, K. 2010. entropy: Entropy and Mutual Information Estimation. R package version 1.1.5. Scholar
Hayek, L. C. and Buzas, M. A. 1997. Surveying Natural Populations. Columbia University Press, 563 p.Google Scholar
Heck, K. L. Jr., van Belle, G., and Simberloff, D. 1975. Explicit calculation of the rarefaction diversity measurement and the determination of sufficient sample size. Ecology, 56:14591461.CrossRefGoogle Scholar
Heim, N. A. 2009. Stability of regional brachiopod diversity structure across the Mississippian/Pennsylvanian boundary. Paleobiology, 35:393412.CrossRefGoogle Scholar
Hill, M. O. 1973. Diversity and evenness: A unifying notation and its consequences. Ecology, 54:427432.CrossRefGoogle Scholar
Hurlbert, S. H. 1971. The nonconcept of species diversity: a critique and alternative parameters. Ecology, 52:577586.CrossRefGoogle Scholar
Jost, L. 2006. Entropy and diversity. Oikos, 113:363375.CrossRefGoogle Scholar
Jost, L. 2007. Partitioning diversity into independent alpha and beta components. Ecology, 88:24272439.CrossRefGoogle ScholarPubMed
Jost, L., DeVries, P., Walla, T., Greeney, H., Chad, A., and Ricotta, C. 2010. Partitioning diversity for conservation analyses. Diversity and Distributions, 16:6576.CrossRefGoogle Scholar
Keylock, C. J. 2005. Simpson diversity and the Shannon-Wiener index as special cases of a generalized entropy. Oikos, 109:203207.CrossRefGoogle Scholar
Lande, R. 1996. Statistics and partitioning of species diversity and similarity among multiple communities. Oikos, 76:513.CrossRefGoogle Scholar
Layou, K. M. 2007. A quantitative null model of additive diversity' partitioning: examining the response of beta diversity to extinction. Paleobiology, 33:116124.CrossRefGoogle Scholar
Legendre, P., Borcard, D., and Peres-Neto, P. R. 2005. Analyzing beta diversity: partitioning the spatial variation of community composition data. Ecological Monographs, 75:435450.CrossRefGoogle Scholar
Legendre, P. and Legendre, L. 1998. Numerical Ecology, 2nd English ed., Elsevier Science BV. Amsterdam, 853 p.Google Scholar
Legendre, P., Mi, X., Ren, H., Ma, K., Yu, M., Sun, I-F., and He, F. 2009. Partitioning beta diversity in a subtropical broadleaved forest of China. Ecology, 90:663674.CrossRefGoogle Scholar
Levins, R. 1968. Evolution in Changing Environments. Princeton University Press, Princeton, New Jersey, 132 p.Google Scholar
Lewontin, R. C. 1972. The apportionment of human diversity. Evolutionary Biology, 6:381398.Google Scholar
MacArthur, R. H. 1965. Patterns of species diversity. Biological Reviews, 40:510533.CrossRefGoogle Scholar
MacArthur, R., Recher, H., and Cody, M. 1966. On the relation between habitat selection and species diversity. The American Naturalist, 98:387, 397.CrossRefGoogle Scholar
Magurran, A. E. 2004, Measuring Biological Diversity. Blackwell Publishing, Oxford, 256 p.Google Scholar
Margalef, D. R. 1958. Information theory in ecology. General Systems, 3: 3671.Google Scholar
Merriam-Webster, . 2003. Merriam-Webster's Collegiate Dictionary, 11th ed. Merriam-Webster, Inc., 1664 p.Google Scholar
Oksanen, J., Blanchet, F. G., Kindt, R., Legendre, P., O'Hara, R. B., Simpson, G. L., Solymos, P., Henry, M., Stevens, H., and Wagner, H. 2010. vegan: Community Ecology Package. R package version 1.17-2. Scholar
Olszewski, T. D. 2004. A unified mathematical framework for the measurement of richness and evenness within and among multiple communities. Oikos, 104:377387.CrossRefGoogle Scholar
Olszewski, T. D. and Kidwell, S. M. 2007. The preservation fidelity of evenness in mollscan death assemblages. Paleobiology, 33:123.CrossRefGoogle Scholar
Patzkowsky, M. E. and Holland, S. M. 2007. Diversity partitioning of a Late Ordovician marine biotic invasion: controls on diversity in regional ecosystems. Paleobiology, 29:545560.2.0.CO;2>CrossRefGoogle Scholar
Pielou, E. C. 1975. Ecological Diversity. Wiley, New York, 176 p.Google Scholar
R Development Core Team. 2010. R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. ISBN 3-900051-07-0, URL ScholarPubMed
Ricotta, C. 2005. On hierarchical diversity decomposition. Journal of Vegetation Science, 16:223226.CrossRefGoogle Scholar
Routledge, R. D. 1977. On Whittaker's components of diversity. Ecology, 58:11201127.CrossRefGoogle Scholar
Scarponi, D. and Kowalewski, M. 2007. Sequence stratigraphic anatomy of diversity patterns: Late Quaternary benthic mollusks of the Po Plain, Italy. Palaios, 22:296305.CrossRefGoogle Scholar
Schneider, T. D. 2008. Information theory primer with an appendix on logarithms., version 2.64, downloaded 8 January 2010.Google Scholar
Sepkoski, J. J. Jr. 1988. Alpha, beta, or gamma: where does all the diversity go? Paleobiology, 14:221234.CrossRefGoogle ScholarPubMed
Shannon, C. E. 1948. A mathematical theory of communication. Bell System Technical Journal, 27:379423 & 623–656.CrossRefGoogle Scholar
Shen, T.-J., Chao, A., and Lin, C.-F. 2003. Predicting the number of new species in further taxonomic sampling. Ecology, 84:798804.CrossRefGoogle Scholar
Simberloff, D. S. 1972. Properties of the rarefaction diversity measurement. The American Naturalist 106:414418.CrossRefGoogle Scholar
Tipper, J. C. 1979, Rarefaction and rarefiction – the use and abuse of a method in paleoecology. Paleobiology, 5:423434.CrossRefGoogle Scholar
Tribus, M. 1961. Thermodynamics and Thermostatics: An Introduction to Energy, Information and States of Matter, with Engineering Applications. D. Van Nostrand Company Inc., New York.Google Scholar
Tomasnových, A. and Kidwell, S. M. 2009. Fidelity of variation in species composition and diversity partitioning by death assemblages: time-averaging transfers diversity from beta to alpha levels. Paleobiology, 35:94118.CrossRefGoogle Scholar
Ugland, K. I., Gray, J. S., and Ellingsen, K. E. 2003. The species-accumulation curve and estimation of species richness. Journal of Animal Ecology, 72:888897.CrossRefGoogle Scholar
Whittaker, R. H. 1960. Vegetation of the Siskiyou Mountains, Oregon and California. Ecological Monographs, 30:279338.CrossRefGoogle Scholar
Whittaker, R. H. 1972. Evolution and measurement of species diversity. Taxon, 21:213251.CrossRefGoogle Scholar
Whittaker, R. H. 1977. Evolution of species diversity in land communities. Evolutionary Biology, 10:167.Google Scholar
Whittaker, R. J., Willis, K. J., and Field, R. 2001. Scale and species richness: towards a general, hierarchical theory of species diversity. Journal of Biogeography, 28:453470.CrossRefGoogle Scholar
Wilson, M. V. and Shmida, A. 1984. Measuring beta diversity with presence-absence data. Journal of Ecology, 72:10551064.CrossRefGoogle Scholar
Vavrek, M. J. and Larsson, H. C. E. 2010. Low beta diversity' of Maastrichtian dinosaurs of North America. Proceedings of the National Academy of Sciences, USA, 107:82658268.CrossRefGoogle Scholar
Veech, J. A., Summerville, K. S., Crist, T. O., and Gehring, J. C. 2002. The additive partitioning of species diversity: recent revival of an old idea. Oikos, 99:39.CrossRefGoogle Scholar
Cited by

Send article to Kindle

To send this article to your Kindle, first ensure is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

Note you can select to send to either the or variations. ‘’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Diversity Partitioning Using Shannon's Entropy and its Relationship to Rarefaction
Available formats

Send article to Dropbox

To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

Diversity Partitioning Using Shannon's Entropy and its Relationship to Rarefaction
Available formats

Send article to Google Drive

To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

Diversity Partitioning Using Shannon's Entropy and its Relationship to Rarefaction
Available formats

Reply to: Submit a response

Please enter your response.

Your details

Please enter a valid email address.

Conflicting interests

Do you have any conflicting interests? *