Skip to main content Accessibility help
×
Home

Invasive Plant Researchers Should Calculate Effect Sizes, Not P-Values

  • Matthew J. Rinella (a1) and Jeremy J. James (a2)

Abstract

Null hypothesis significance testing (NHST) forms the backbone of statistical inference in invasive plant science. Over 95% of research articles in Invasive Plant Science and Management report NHST results such as P-values or statistics closely related to P-values such as least significant differences. Unfortunately, NHST results are less informative than their ubiquity implies. P-values are hard to interpret and are regularly misinterpreted. Also, P-values do not provide estimates of the magnitudes and uncertainties of studied effects, and these effect size estimates are what invasive plant scientists care about most. In this paper, we reanalyze four datasets (two of our own and two of our colleagues; studies put forth as examples in this paper are used with permission of their authors) to illustrate limitations of NHST. The re-analyses are used to build a case for confidence intervals as preferable alternatives to P-values. Confidence intervals indicate effect sizes, and compared to P-values, confidence intervals provide more complete, intuitively appealing information on what data do/do not indicate.

    • Send article to Kindle

      To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

      Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

      Find out more about the Kindle Personal Document Service.

      Invasive Plant Researchers Should Calculate Effect Sizes, Not P-Values
      Available formats
      ×

      Send article to Dropbox

      To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

      Invasive Plant Researchers Should Calculate Effect Sizes, Not P-Values
      Available formats
      ×

      Send article to Google Drive

      To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

      Invasive Plant Researchers Should Calculate Effect Sizes, Not P-Values
      Available formats
      ×

Copyright

Corresponding author

Corresponding author's E-mail: matt.rinella@ars.usda.gov

References

Hide All
Anderson, D. R., Burnham, K. P., and Thompson, W. L. 2000. Null hypothesis testing: problems, prevalence and an alternative. J. Wildl. Manag 64:912923.
Anderson, D. R., Link, W. A., Johnson, D. H., and Burnham, K. P. 2001. Suggestions for presenting the results of data analysis. J. Wildl. Manag 65:373378.
Bates, J. D. 2005. Herbaceous response to cattle grazing following juniper cutting in Oregon. Rangeland Ecol. Manag 58:225233.
Berger, J. O. and Berry, D. A. 1988. Statistical analysis and the illusion of objectivity. Am. Sci 76:159165.
Berger, J. O. and Sellke, T. 1987. Testing a point null hypothesis: the irreconcilability of P values and evidence. J. Am. Statistical Assoc 82:112122.
Berry, D. A. and Lindgren, B. W. 1996. Statistics, Theory and Methods. Belmont, CA Wadsworth. 702.
Casella, G. and Berger, R. L. 1987. Reconciling Bayesian and frequentist evidence in the one-sided testing problem (with comments). J. Am. Statistical Assoc 82:106139.
Cohen, J. 1994. The earth is round (p <.05). Am. Psychologist 49:9971003.
Cumming, G. and Finch, S. 2001. A primer on the understanding, use, and calculation of confidence intervals that are based on central and noncentral distributions. Educ. Psychol. Meas 61:532574.
D'Antonio, C. M. and Levine, J. M. 1999. Elton revisited: a review of evidence linking diversity and invasibility. Oikos 87:1526.
Diamond, G. A. and Forrester, J. S. 1983. Clinical trials and statistical verdicts: probable grounds for appeal. Ann. Internal Med 98:385394.
Falk, R. and Greenbaum, C. W. 1995. Significance tests die hard. The amazing persistence of a probabilistic misconception. Theory Psychol 5:7598.
Fidler, F., Burgman, M. A., Cumming, G., Buttrose, R., and Thomason, N. 2006. Impact of criticism of null-hypothesis significance testing on statistical reporting practices in conservation biology. Conserv. Biol 20:15391544.
Fisher, R. A. 1929. The statistical method in psychical research. Proc. Soc. Psychical Res 39:189192.
Gelman, A., Carlin, J. B., Stern, H. S., and Rubin, D. B. 2004. Bayesian data analysis. Boca Raton, FL Chapman & Hall/CRC. 668.
Guthery, F. S., Lusk, J. J., and Peterson, M. J. 2001. The fall of null hypothesis: liabilities and opportunities. J. Wildl. Manag 65:379384.
Heitschmidt, R. K. and Vermeire, L. T. 2006. Can abundant summer precipitation counter losses in herbage production caused by spring drought. Rangeland Ecol. Manag 59:392399.
Hubbard, R. and Lindsay, R. M. 2008. Why P values are not a useful measure of evidence in statistical significance testing. Theory Psychol 18:6988.
James, J. J., Davies, K. W., Sheley, R. L., and Aanderud, Z. T. 2008. Linking nitrogen partitioning and species abundance to invasion resistance in the Great Basin. Oecologia 156:637648.
Kirk, R. E. 1996. Practical significance: a concept whose time has come. Educ. Psych. Meas 56:741745.
Martinez-Abrain, A. 2007. Are there any differences? A non-sensical question in ecology. Acta Ecol. Int. J. Ecol 32:203206.
Nagele, P. 2001. Misuse of standard error of the mean (SEM) when reporting variability of a sample. A critical evaluation of four anaesthesia journals. Br. J. Anaesth 90:514516.
Nakagawa, S. and Cuthill, I. C. 2007. Effect size, confidence interval and statistical significance: a practical guide for biologists. Biol. Rev 82:591605.
Nelder, J. A. 1999. From statistics to statistical science. The Statistician 48:257269.
Neter, J., Kutner, M. H., Nachtsheim, C. J., and Wasserman, W. 1996. Applied linear statistical models. New York Irwin. 1408.
Nickerson, R. S. 2000. Null hypothesis significance testing: a review of an old and continuing controversy. Psych. Methods 5:241301.
Rinella, M. J., Jacobs, J. S., Sheley, R. L., and Borkowski, J. J. 2001. Spotted knapweed response to season and frequency of mowing. J. Range Manag 54:5256.
Robert, C. P. 2001. The Bayesian Choice. Paris, France Springer. 604.
Rosenthal, R. and Rubin, D. B. 1994. The counternull value of an effect size. Psychol. Sci 5:329334.
SAS 1999. Statistical software. Version 8.0. Cary, NC SAS Institute.
Sellke, T., Bayarri, M. J., and Berger, J. O. 2001. Calibration of p values for testing precise null hypotheses. Am. Statistician 55:6271.
Stephens, P. A., Buskirk, S. W., and Martinez del Rio, C. 2007. Inferences in ecology and evolution. Trends Ecol. Evol 22:192197.
Tukey, J. W. 1991. The philosophy of multiple comparisons. Statistical Sci 6:100116.

Keywords

Invasive Plant Researchers Should Calculate Effect Sizes, Not P-Values

  • Matthew J. Rinella (a1) and Jeremy J. James (a2)

Metrics

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed