Hostname: page-component-848d4c4894-r5zm4 Total loading time: 0 Render date: 2024-07-04T03:23:47.475Z Has data issue: false hasContentIssue false

CE Workshop 09: Cast Aside Traditional Notions of Statistical Significance, and Focus Instead on Characterizing the Magnitude of Effects that are Clinically or Scientifically Relevant

Published online by Cambridge University Press:  21 December 2023

Rights & Permissions [Opens in a new window]

Abstract & learning objectives

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

There is an ongoing debate among statisticians and discipline scientists about the consequences of our persistent, dogmatic reliance on evaluating all statistical results as meaningful if and only if "p<0.05," regardless of context. This was never the intended goal of Ronald Fisher, nevertheless scientists have adopted it as a convenience, and the decades long dependence on "p<0.05" has had important negative consequences. In this presentation, I review common misconceptions about interpreting p-values, why we should consider de-emphasizing p-values, and why scientists should rely more on practical, clinical, or scientifically meaningful differences over arbitrary cut-offs. I will present several different metrics for evaluating and reporting effect magnitude, and whether or not data support the null vs. alternative hypothesis, under the frequentist paradigm, how Bayesian methods can augment or replace frequentist analyses, and a few options that help to clarify how important a finding may be. Throughout this talk, I advocate that discipline scientists take charge of sharing scientific results that are not based merely on arbitrary p-value cutoffs and other default logic, but instead based on their content expertise, in light of all of the specific relevant aspects of experimental design and experimental data, balancing the consequences of Type I vs Type II errors appropriately, and focusing on characterizing effects, rather than dichotomizing research into only two categories of importance (significant vs. not).

Upon conclusion of this course, learners will be able to:

  1. 1. Discuss what p-values mean and how they are commonly misinterpreted.

  2. 2. Explain the leading arguments promoted by the American Statistical Association with regard to why science should carefully reconsider if and how p-values should continue to dominate our decisions about what research should be published, and how scientists should be evaluating its worth.

  3. 3. Apply new practices in how to evaluate and publish their own research, as well as how to evaluate research appearing in peer-reviewed journals, whether as consumers, reviewers, or editors.

Type
Abstract
Copyright
Copyright © INS. Published by Cambridge University Press, 2023