Hostname: page-component-77c89778f8-n9wrp Total loading time: 0 Render date: 2024-07-18T04:56:22.291Z Has data issue: false hasContentIssue false
  • English
  • Français

How Trustworthy Is the Scientific Literature in Industrial and Organizational Psychology?

Published online by Cambridge University Press:  07 January 2015

Sven Kepes  and
Michael A. McDaniel
Sven Kepes*
Affiliation:
Virginia Commonwealth University
Michael A. McDaniel
Affiliation:
Virginia Commonwealth University
*
E-mail: skepes@vcu.edu, Address: Virginia Commonwealth University, 301 West Main Street, PO Box 844000, Richmond, VA 23284
Get access Rights & Permissions [Opens in a new window]

Abstract

The trustworthiness of research findings has been questioned in many domains of science. This article calls for a review of the trustworthiness of the scientific literature in industrial–organizational (I–O) psychology and a reconsideration of common practices that may harm the credibility of our literature. We note that most hypotheses in I–O psychology journals are confirmed. Thus, we are either approaching omniscience or our journals are publishing an unrepresentative sample of completed research. We view the latter explanation as more likely. We review structural problems in the publication process and in the conduct of research that is likely to promote a distortion of scientific knowledge. We then offer recommendations to make the I–O literature more accurate and trustworthy.

Type
Focal Article
Information
Industrial and Organizational Psychology , Volume 6 , Issue 3 , September 2013 , pp. 252 - 268
Copyright
Copyright © Society for Industrial and Organizational Psychology 2013

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Academy of Management Journal. (2012). Information for Contributors. Retrieved from http://aom.org/Publications/AMJ/Information-for-Contributors.aspxGoogle Scholar
Aguinis, H., Dalton, D. R., Bosco, F. A., Pierce, C. A., & Dalton, C. M. (2011). Meta-analytic choices and judgment calls: Implications for theory building and testing, obtained effect sizes, and scholarly impact. Journal of Management, 37, 5–38. doi: 10.1177/0149206310377113CrossRefGoogle Scholar
Allison, P. D. (1978). Measures of inequality. American Sociological Review, 43, 865–880. doi: 10.2307/2094626CrossRefGoogle Scholar
American Psychological Association. (2002). Ethical principles of psychologists and code of conduct. American Psychologist, 57, 1060–1073. doi: 10.1037/0003-066x.57.12.1060CrossRefGoogle Scholar
American Psychological Association. (2008). Reporting standards for research in psychology: Why do we need them? What might they be? American Psychologist, 63, 839–851. doi: 810.1037/0003-1066X.1063.1039.1839CrossRefGoogle Scholar
American Psychological Association. (2010). Publication manual of the American Psychological Association (6th ed.). Washington, DC: American Psychological Association.Google Scholar
Aytug, Z. G., Rothstein, H. R., Zhou, W., & Kern, M. C. (2012). Revealed or concealed? Transparency of procedures, decisions, and judgment calls in meta-analyses. Organizational Research Methods, 15, 103–133. doi: 10.1177/1094428111403495Google Scholar
Baggerly, K. (2010). Disclose all data in publications. Nature, 467, 401. doi: 10.1038/467401bCrossRefGoogle ScholarPubMed
Banks, G. C., Kepes, S., & Banks, K. P. (2012). Publication bias: The antagonist of meta-analytic reviews and effective policy making. Educational Evaluation and Policy Analysis, 34, 259–277. doi: 10.3102/0162373712446144Google Scholar
Banks, G. C., Kepes, S., & McDaniel, M. A. (2012). Publication bias: A call for improved meta-analytic practice in the organizational sciences. International Journal of Selection and Assessment, 20, 182–196. doi: 10.1111/j.1468-2389.2012.00591.xCrossRefGoogle Scholar
Banks, G. C., & McDaniel, M. A. (2011). The kryptonite of evidence-based I–O psychology. Industrial and Organizational Psychology: Perspectives on Science and Practice, 4, 40–44. doi: 10.1111/j.1754-9434.2010.01292.xGoogle Scholar
Bargh, J. A., Chen, M., & Burrows, L. (1996). Automaticity of social behavior: Direct effects of trait construct and stereotype activation on action. Journal of Personality and Social Psychology, 71, 230–244. doi: 10.1037/0022-3514.71.2.230Google Scholar
Bedeian, A. G., Taylor, S. G., & Miller, A. N. (2010). Management science on the credibility bubble: Cardinal sins and various misdemeanors. Academy of Management Learning & Education, 9, 715–725. doi: 10.5465/amle.2010.56659889Google Scholar
Begley, C. G., & Ellis, L. M. (2012). Drug development: Raise standards for preclinical cancer research. Nature, 483, 531–533. doi: 10.1038/483531aGoogle Scholar
Bem, D. J. (2011). Feeling the future: Experimental evidence for anomalous retroactive influences on cognition and affect. Journal of Personality and Social Psychology, 100, 407–425. doi: 10.1037/a0021524CrossRefGoogle ScholarPubMed
Berenson, A. (2005, May 31). Despite vow, drug makers still withhold data. New York Times, p. A1. Retrieved from http://www.nytimes.com/2005/05/31/business/31trials.html.Google Scholar
Berlin, J. A., & Ghersi, D. (2005). Preventing publication bias: Registries and prospective meta-analysis. In Rothstein, H. R., Sutton, A. J., & Borenstein, M. (Eds.), Publication bias in meta analysis: Prevention, assessment, and adjustments. West Sussex, England: Wiley.Google Scholar
Braxton, J. M. (1999). Perspectives on scholarly misconduct in the sciences. Columbus, OH: Ohio State University Press.Google Scholar
Carey, B. (2011, November 3). Fraud case seen as a red flag for psychology research. New York Times, p. 3. Retrieved from http://www.nytimes.com/2011/11/03/health/research/noted-dutch-psychologist-stapel-accused-of-research-fraud.htmlGoogle Scholar
Chalmers, T. C., Levin, H., Sacks, H. S., Reitman, D., Berrier, J., & Nagalingam, R. (1987). Meta-analysis of clinical trials as a scientific discipline. I: Control of bias and comparison with large co-operative trials. Statistics in Medicine, 6, 315–325. doi: 10.1002/sim.4780060320CrossRefGoogle ScholarPubMed
Crane, D. (1967). The gatekeepers of science: Some factors affecting the selection of articles for scientific journals. The American Sociologist, 2, 195–201. doi: 10.2307/27701277Google Scholar
Crocker, J., & Cooper, M. L. (2011). Addressing scientific fraud. Science, 334, 1182. doi: 10.1126/science.1216775Google Scholar
Dalton, D. R., Aguinis, H., Dalton, C. M., Bosco, F. A., & Pierce, C. A. (2012). Revisiting the file drawer problem in a meta-analysis: An assessment of published and nonpublished correlation matrices. Personnel Psychology, 65, 221–249. doi: 10.1111/j.1744-6570.2012.01243.xGoogle Scholar
Darwin, C. (1981/1871). The descent of man and selection in relation to sex. London, England: Princeton University Press.CrossRefGoogle Scholar
Dickersin, K. (1990). The existence of publication bias and risk factors for its occurrence. Journal of the American Medical Association, 263, 1385–1389. doi: 10.1001/jama.1990.03440100097014Google Scholar
Dickersin, K. (2005). Publication bias: Recognizing the problem, understandings its origins and scope, and preventing harm. In Rothstein, H. R., Sutton, A. J., & Borenstein, M. (Eds.), Publication bias in meta-analysis: Prevention, assessment, and adjustments (pp. 11–34). West Sussex, England: Wiley.Google Scholar
Dickersin, K., Manheimer, E., Wieland, S., Robinson, K. A., Lefebvre, C., McDonald, S., & Group, C. D. (2002). Development of the Cochrane Collaboration's central register of controlled clinical trials. Evaluation & the Health Professions, 25, 38–64. doi: 10.1177/016327870202500104Google ScholarPubMed
Doyen, S., Klein, O., Pichon, C.-L., & Cleeremans, A. (2011). Behavioral priming: It's all in the mind, but whose mind? PLoS ONE, 7, e29081. doi: 10.1371/journal.pone.0029081CrossRefGoogle Scholar
Easterbrook, P. J., Gopalan, R., Berlin, J. A., & Matthews, D. R. (1991). Publication bias in clinical research. The Lancet, 337, 867–872. doi: 10.1016/0140-6736(91)90201-yGoogle Scholar
Evangelou, E., Trikalinos, T. A., & Ioannidis, J. P. (2005). Unavailability of online supplementary scientific information from articles published in major journals. The FASEB Journal, 19, 1943–1944. doi: 10.1096/fj.05-4784lsfGoogle Scholar
Fanelli, D. (2010a). Do pressures to publish increase scientists' bias? An empirical support from US states data. PLoS ONE, 5, e10271. doi: 10.1371/journal.pone.0010271Google Scholar
Fanelli, D. (2010b). “Positive” results increase down the hierarchy of the sciences. PLoS ONE, 5, e10068. doi: 10.1371/journal.pone.0010068CrossRefGoogle ScholarPubMed
Fanelli, D. (2012). Negative results are disappearing from most disciplines and countries. Scientometrics, 90, 891–904. doi: 10.1007/s11192-011-0494-7Google Scholar
Ferguson, C. J., & Brannick, M. T. (2012). Publication bias in psychological science: Prevalence, methods for identifying and controlling, and implications for the use of meta-analyses. Psychological Methods, 17, 120–128. doi: 10.1037/a0024445Google Scholar
Ferguson, C. J., & Heene, M. (2012). A vast graveyard of undead theories: Publication bias and psychological science's aversion to the null. Perspectives on Psychological Science, 7, 555–561. doi: 10.1177/1745691612459059Google Scholar
Francis, G. (2012). Too good to be true: Publication bias in two prominent studies from experimental psychology. Psychonomic Bulletin & Review, 19, 151–156. doi: 10.3758/s13423-012-0227-9Google Scholar
Fuchs, H. M., Jenny, M., & Fiedler, S. (2012). Psychologists are open to change, yet wary of rules. Perspectives on Psychological Science, 7, 639–642. doi: 10.1177/1745691612459521Google Scholar
Gallup Organization. (2008). Observing and reporting suspected misconduct in biomedical research. Retrieved from http://ori.hhs.gov/sites/default/files/gallup_finalreport.pdfGoogle Scholar
Giner-Sorolla, R. (2012). Science or art? How aesthetic standards grease the way through the publication bottleneck but undermine science. Perspectives on Psychological Science, 7, 562–571. doi: 10.1177/1745691612457576Google Scholar
Goldacre, B. (2012). Bad pharma: How drug companies mislead doctors and harm patients. London, England: Fourth Estate.Google Scholar
Goldstein, D. (2010). On fact and fraud: Cautionary tales from the front lines of science. Princeton, NJ: Princeton University Press.Google Scholar
Gomez-Mejia, L. R., & Balkin, D. B. (1992). Determinants of faculty pay: An agency theory perspective. Academy of Management Journal, 35, 921–955. doi: 10.2307/256535Google Scholar
Greenwald, A. G. (1975). Consequences of prejudice against the null hypothesis. Psychological Bulletin, 82, 1–20. doi: 10.1037/h0076157Google Scholar
Greenwald, A. G. (2012). There is nothing so theoretical as a good method. Perspectives on Psychological Science, 7, 99–108. doi: 10.1177/1745691611434210CrossRefGoogle ScholarPubMed
Hambrick, D. C. (2007). The field of management's devotion to theory: Too much of a good thing? Academy of Management Journal, 50, 1348–1352. doi: 10.5465/amj.2007.28166119Google Scholar
Hanson, B., Sugden, A., & Alberts, B. (2011). Making data maximally available. Science, 331, 649. doi: 10.1126/science.1203354Google Scholar
Harrison, D. A., & Klein, K. J. (2007). What's the difference? Diversity constructs as separation, variety, or disparity in organizations. The Academy of Management Review, 32, 1199–1228. doi: 10.2307/20159363Google Scholar
Ioannidis, J. P. A. (2005). Why most published research findings are false. PLoS Medicine, 2, e124. doi: 10.1371/journal.pmed.0020124Google Scholar
Ioannidis, J. P. A. (2006). Concentration of the most-cited papers in the scientific literature: Analysis of journal ecosystems. PLoS ONE, 1, e5. doi: 10.1371/journal.pone.0000005Google Scholar
Ioannidis, J. P. A. (2012). Why science is not necessarily self-correcting. Perspectives on Psychological Science, 7, 645–654. doi: 10.1177/1745691612464056CrossRefGoogle Scholar
Ioannidis, J. P. A., Allison, D. B., Ball, C. A., Coulibaly, I., Cui, X., Culhane, A. C., … van Noort, V. (2009). Repeatability of published microarray gene expression analyses. Nature Genetics, 41, 149–155. doi: 10.1038/ng.295Google Scholar
Jasny, B. R., Chin, G., Chong, L., & Vignieri, S. (2011). Again, and again, and again…. Science, 334, 1225. doi: 10.1126/science.334.6060.1225Google Scholar
Jefferson, T. (1998). Redundant publication in biomedical sciences: Scientific misconduct or necessity? Science and Engineering Ethics, 4, 135–140. doi: 10.1007/s11948-998-0043-9CrossRefGoogle ScholarPubMed
Kepes, S., Banks, G. C., McDaniel, M. A., & Whetzel, D. L. (2012). Publication bias in the organizational sciences. Organizational Research Methods, 15, 624–662. doi: 10.1177/1094428112452760Google Scholar
Kepes, S., Banks, G., & Oh, I.-S. (in press). Avoiding bias in publication bias research: The value of “null” findings. Journal of Business and Psychology. doi: 10.1007/s10869-012-9279-0Google Scholar
Kepes, S., McDaniel, M. A., Brannick, M. T., & Banks, G. C. (2013). Meta-analytic reviews in the organizational sciences: Two meta-analytic schools on the way to MARS (the Meta-analytic Reporting Standards). Journal of Business and Psychology, 28, 123–143. doi: 10.1007/s10869-013-9300-2CrossRefGoogle Scholar
Kerr, S. (1975). On the folly of rewarding A, while hoping for B. Academy of Management Journal, 18, 769–783. doi: 10.2307/255378Google Scholar
Kerr, N. L. (1998). HARKing: Hypothesizing after the results are known. Personality and Social Psychology Review, 2, 196–217. doi: 10.1207/s15327957pspr0203_4Google Scholar
Koole, S. L., & Lakens, D. (2012). Rewarding replications: A sure and simple way to improve psychological science. Perspectives on Psychological Science, 7, 608–614. doi: 10.1177/1745691612462586Google Scholar
Kozlowski, S. W. J. (2009). Editorial. Journal of Applied Psychology, 94, 1–4. doi: 10.1037/a0014990Google Scholar
Krimsky, S. (2006). Publication bias, data ownership, and the funding effect in science: Threats to the integrity of biomedical research. In Wagner, W., & Steinzor, R. (Eds.), Rescuing science from politics: Regulation and the distortion of scientific research (pp. 61–85). New York, NY: Cambridge University Press.Google Scholar
LaFollette, M. C. (1992). Stealing into print: Fraud, plagiarism, and misconduct in scientific publishing. Los Angeles, CA: University of California Press.Google Scholar
Laine, C., Horton, R., DeAngelis, C. D., Drazen, J. M., Frizelle, F. A., Godlee, F., … Verheugt, F. W. A. (2007). Clinical trial registration: Looking back and moving ahead. New England Journal of Medicine, 356, 2734–2736. doi: 10.1056/NEJMe078110Google Scholar
LeBel, E. P., & Peters, K. R. (2011). Fearing the future of empirical psychology: Bem's (2011) evidence of psi as a case study of deficiencies in modal research practice. Review of General Psychology, 15, 371–379. doi: 10.1037/a0025172CrossRefGoogle Scholar
Lehrer, J. (2010). The truth wears off. New Yorker, 86, 52–57.Google Scholar
Leung, K. (2011). Presenting post hoc hypotheses as a priori: Ethical and theoretical issues. Management and Organization Review, 7, 471–479. doi: 10.1111/j.1740-8784.2011.00222.xGoogle Scholar
Lewin, K. (1952). Field theory in social science: Selected theoretical papers by Kurt Lewin. London, England: Tavistock.Google Scholar
Liberati, A. (1992). Publication bias and the editorial process. Journal of the American Medical Association, 267, 2891. doi: 10.1001/jama.1992.03480210049017Google Scholar
Makel, M. C., Plucker, J. A., & Hegarty, B. (2012). Replications in psychology research: How often do they really occur? Perspectives on Psychological Science, 7, 537–542. doi: 10.1177/1745691612460688CrossRefGoogle ScholarPubMed
McDaniel, M. A., McKay, P., & Rothstein, H. R. (2006). Publication bias and racial effects on job performance: The elephant in the room. Paper presented at the annual meeting of the Society for Industrial and Organizational Psychology, Dallas, TX.Google Scholar
McDaniel, M. A., Rothstein, H. R., & Whetzel, D. L. (2006). Publication bias: A case study of four test vendors. Personnel Psychology, 59, 927–953. doi: 10.1111/j.1744-6570.2006.00059.xGoogle Scholar
McDaniel, M. A., & Whetzel, D. L. (2005). Situational judgment test research: Informing the debate on practical intelligence theory. Intelligence, 33, 515–525. doi: 10.1016/j.intell.2005.02.001Google Scholar
Merton, R. K. (1942). Science and technology in a democratic order. Journal of Legal and Political Sociology, 1, 115–126.Google Scholar
Merton, R. K. (1973). The sociology of science: Theoretical and empirical investigations. Chicago, IL: The University of Chicago Press.Google Scholar
Murphy, J. R. (2004). Statistical errors in immunologic research. The Journal of allergy and clinical immunology, 114, 1259–1263.Google Scholar
Murtaugh, P. A. (2002). Journal quality, effect size, and publication bias in meta-analysis. Ecology, 83, 1162–1166. doi: 10.1890/0012-9658(2002)083[1162:jqesap]2.0.co;2Google Scholar
Neuliep, J. W., & Crandall, R. (1990). Editorial bias against replication research. Journal of Social Behavior & Personality, 5, 85–90.Google Scholar
Neuliep, J. W., & Crandall, R. (1993). Reviewer bias against replication research. Journal of Social Behavior & Personality, 8, 21–29.Google Scholar
Nickerson, R. S. (1998). Confirmation bias: A ubiquitous phenomenon in many guises. Review of General Psychology, 2, 175–220. doi: 10.1037/1089-2680.2.2.175Google Scholar
Nosek, B. A., Spies, J. R., & Motyl, M. (2012). Scientific utopia: II. Restructuring incentives and practices to promote truth over publishability. Perspectives on Psychological Science, 7, 615–631. doi: 10.1177/1745691612459058Google Scholar
Open Science Collaboration. (2012). An open, large-scale, collaborative effort to estimate the reproducibility of psychological science. Perspectives on Psychological Science, 7, 657–660. doi: 10.1177/1745691612462588Google Scholar
Orlitzky, M. (2012). How can significance tests be deinstitutionalized? Organizational Research Methods, 15, 199–228. doi: 10.1177/1094428111428356Google Scholar
Pace, V. L., & Brannick, M. T. (2010). How similar are personality scales of the “same” construct? A meta-analytic investigation. Personality and Individual Differences, 49, 669–676. doi: 10.1016/j.paid.2010.06.014CrossRefGoogle Scholar
Park, R. L. (2001). Voodoo science: The road from foolishness to fraud. Oxford, England: Oxford University Press.Google Scholar
Pashler, H., & Harris, C. R. (2012). Is the replicability crisis overblown? Three arguments examined. Perspectives on Psychological Science, 7, 531–536. doi: 10.1177/1745691612463401Google Scholar
Popper, K. R. (1959). The logic of scientific discovery. Oxford, England: Basic Books.Google Scholar
Porter, T. M. (1992). Quantification and the accounting ideal in science. Social Studies of Science, 22, 633–652. doi: 10.1177/030631292022004004Google Scholar
Rauscher, F. H., Shaw, G. L., & Ky, C. N. (1993). Music and spatial task performance. Nature, 365, 611. doi: 10.1038/365611a0Google Scholar
Reich, E. S. (2010). Plastic fantastic: How the biggest fraud in physics shook the scientific world. New York, NY: Palgrave Macmillan.Google Scholar
Renkewitz, F., Fuchs, H. M., & Fiedler, S. (2011). Is there evidence of publication biases in JDM research? Judgment and Decision Making, 6, 870–881.Google Scholar
Ritchie, S. J., Wiseman, R., & French, C. C. (2012). Failing the future: Three unsuccessful attempts to replicate Bem's 'retroactive facilitation of recall' effect. PLoS ONE, 7. doi: 10.1371/journal.pone.0033423Google Scholar
Roberson, Q. M., Sturman, M. C., & Simons, T. L. (2007). Does the measure of dispersion matter in multilevel research? A comparison of the relative performance of dispersion indexes. Organizational Research Methods, 10, 564–588. doi: 10.1177/1094428106294746Google Scholar
Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86, 638–641. doi: 10.1037/0033-2909.86.3.638Google Scholar
Rothstein, H. (2012). Accessing relevant literature. In Cooper, H. M. (Ed.), APA handbook of research methods in psychology: Vol. 1. Foundations, planning, measures, and psychometrics (pp. 133–144). Washington, DC: American Psychological Association.Google Scholar
Rothstein, H. R., Sutton, A. J., & Borenstein, M. (2005a). Publication bias in meta-analyses. In Rothstein, H. R., Sutton, A. J., & Borenstein, M. (Eds.), Publication bias in meta analysis: Prevention, assessment, and adjustments (pp. 1–7). West Sussex, England: Wiley.Google Scholar
Rothstein, H. R., Sutton, A. J., & Borenstein, M. (2005b). Publication bias in meta-analysis: Prevention, assessment, and adjustments. West Sussex, England: Wiley.Google Scholar
Rouder, J. N., & Morey, R. D. (2011). A Bayes factor meta-analysis of Bem's ESP claim. Psychonomic Bulletin & Review, 18, 682–689. doi: 10.3758/s13423-011-0088-7Google Scholar
Rupp, D. E. (2011). Ethical issues faced by editors and reviewers. Management and Organization Review, 7, 481–493. doi: 10.1111/j.1740-8784.2011.00227.xGoogle Scholar
Sarawitz, D. (2012). Beware the creeping cracks of bias. Nature, 485, 149. doi: 10.1038/485149aGoogle Scholar
Saul, S. (2008, October 8). Experts conclude Pfizer manipulated studies. New York Times, p. 4. Retrieved from http://www.nytimes.com/2008/10/08/health/research/08drug.htmlGoogle Scholar
Schmidt, S. (2009). Shall we really do it again? The powerful concept of replication is neglected in the social sciences. Review of General Psychology, 13, 90–100. doi: 10.1037/a0015108Google Scholar
Schminke, M. (2009). Editor's comments: The better angels of our time-Ethics and integrity in the publishing process. The Academy of Management Review, 34, 586–591. doi: 10.5465/amr.2009.44882922Google Scholar
Schminke, M., & Ambrose, M. L. (2011). Ethics and integrity in the publishing process: Myths, facts, and a roadmap. Management and Organization Review, 7, 397–406. doi: 10.1111/j.1740-8784.2011.00248.xGoogle Scholar
Schmitt, N. (1989). Editorial. Journal of Applied Psychology, 74, 843–845. doi: 10.1037/h0092216Google Scholar
Schooler, J. W., & Engstler-Schooler, T. Y. (1990). Verbal overshadowing of visual memories: Some things are better left unsaid. Cognitive Psychology, 22, 36–71. doi: 10.1016/0010-0285(90)90003-mGoogle Scholar
Schwab, D. P. (2005). Research methods for organizational studies (2nd ed.). Mahwah, NJ: Erlbaum.Google Scholar
Shields, P. G. (2000). Publication bias is a scientific problem with adverse ethical outcomes: The case for a section for null results. Cancer Epidemiology Biomarkers & Prevention, 9, 771–772.Google Scholar
Shields, P. G., Sellers, T. A., & Rebbeck, T. R. (2009). Null results in brief: Meeting a need in changing times. Cancer Epidemiology Biomarkers & Prevention, 18, 2347. doi: 10.1158/1055-9965.epi-09-0684Google Scholar
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22, 1359–1366. doi: 10.1177/0956797611417632Google Scholar
Sterling, T. D. (1959). Publication decisions and their possible effects on inferences drawn from tests of significance—or vice versa. Journal of the American Statistical Association, 54, 30–34. doi: 10.2307/2282137Google Scholar
Sterling, T. D., & Rosenbaum, W. L. (1995). Publication decisions revisited: The effect of the outcome of statistical tests on the decision to publish and vice versa. American Statistician, 49, 108–112. doi: 10.1080/00031305.1995.10476125Google Scholar
Sternberg, R. J., Forsythe, G. B., Hedlund, J., Horvath, J. A., Wagner, R. K., Williams, W. M.,… Grigorenko, E. L. (2000). Practical intelligence in everyday life. New York, NY: Cambridge University Press.Google Scholar
Strasak, A. M., Zaman, Q., Marinell, G., Pfeiffer, K. P., & Ulmer, H. (2007). The use of statistics in medical research. The American Statistician, 61, 47–55. doi: 10.1198/000313007x170242Google Scholar
Stroebe, W., Postmes, T., & Spears, R. (2012). Scientific misconduct and the myth of self-correction in science. Perspectives on Psychological Science, 7, 670–688. doi: 10.1177/1745691612460687Google Scholar
Sutton, A. J. (2005). Evidence concerning the consequences of publication and related biases. In Rothstein, H. R., Sutton, A. J., & Borenstein, M. (Eds.), Publication bias in meta-analysis: Prevention, assessment, and adjustments (pp. 175–192). West Sussex, England: Wiley.Google Scholar
Sutton, A. J. (2009). Publication bias. In Cooper, H., Hedges, L. V., & Valentine, J. C. (Eds.), The handbook of research synthesis and meta-analysis (2nd ed., pp. 435–452). New York, NY: Russell Sage Foundation.Google Scholar
Sutton, R. I., & Staw, B. M. (1995). What theory is not. Administrative Science Quarterly, 40, 371–384. doi: 10.2307/2393788Google Scholar
Trikalinos, T. A., & Ioannidis, J. P. A. (2005). Assessing the evolution of effect sizes over time. In Rothstein, H. R., Sutton, A. J., & Borenstein, M. (Eds.), Publication bias in meta-analysis: Prevention, assessment and adjustments (pp. 241–259). West Sussex, England: Wiley.Google Scholar
Turner, E. H. (2004). A taxpayer-funded clinical trials registry and results database. PLoS Medicine, 1, e60. doi: 10.1371/journal.pmed.0010060Google Scholar
Uchino, B. N., Thoman, D., & Byerly, S. (2010). Inference patterns in theoretical social psychology: Looking back as we move forward. Social and Personality Psychology Compass, 4, 417–427. doi: 10.1111/j.1751-9004.2010.00272.xGoogle Scholar
Van Dongen, S., & Gangestad, S. W. (2011). Human fluctuating asymmetry in relation to health and quality: A meta-analysis. Evolution and Human Behavior, 32, 380–398. doi: 10.1016/j.evolhumbehav.2011.03.002Google Scholar
Vogel, G. (2011). Psychologist accused of fraud on “astonishing scale.” Science, 334, 579. doi: 10.1126/science.334.6056.579Google Scholar
Wagenmakers, E.-J., Wetzels, R., Borsboom, D., van derMaas, H. L. J., & Kievit, R. A. (2012). An agenda for purely confirmatory research. Perspectives on Psychological Science, 7, 632–638. doi: 10.1177/1745691612463078Google Scholar
Whalen, J., Barrett, D., & Loftus, P. (2012, July 3). Glaxo sets guilty plea, $3 billion settlement. Wall Street Journal, B1. Retrieved from http://online.wsj.com/article/SB10001424052702304299704577502642401041730.htmlGoogle Scholar
Wicherts, J. M., Bakker, M., & Molenaar, D. (2011). Willingness to share research data is related to the strength of the evidence and the quality of reporting of statistical results. PLoS ONE, 6, e26828. doi: 10.1371/journal.pone.0026828Google Scholar
Wicherts, J. M., Borsboom, D., Kats, J., & Molenaar, D. (2006). The poor availability of psychological research data for reanalysis. American Psychologist, 61, 726–728. doi: 10.1037/0003-066x.61.7.726Google Scholar
Witten, D. M., & Tibshirani, R. (2012). Scientific research in the age of omics: The good, the bad, and the sloppy. Journal of the American Medical Informatics Association, 1–3. doi: 10.1136/amiajnl-2012-000972Google Scholar
Wolins, L. (1962). Responsibility for raw data. American Psychologist, 17, 657–658. doi: 10.1037/h0038819Google Scholar
Yong, E. (2012a). Replication studies: Bad copy. Nature, 485, 298–300. doi: 10.1038/485298aGoogle Scholar
Yong, E. (2012b). Uncertainty shrouds psychologist's resignation. Nature. doi: 10.1038/nature.2012.10968Google Scholar