No CrossRef data available.
Article contents
On the potential distortions of highly cited papers in emerging research fields: A critical appraisal
Published online by Cambridge University Press: 15 July 2019
Abstract
Citation-based metrics are increasingly used as a proxy to define representative, considerable, or significant papers. We challenge this belief by taking into account factors that may play a role in providing citations to a manuscript and whether/how those highly cited studies could shape a scientific field. A different approach to summarisation of relevant core publications within a topic is proposed.
- Type
- Open Peer Commentary
- Information
- Copyright
- Copyright © Cambridge University Press 2019
References
Berer, K., Mues, M., Koutrolos, M., Rasbi, Z. Al, Boziki, M., Johner, C., Wekerle, H. & Krishnamoorthy, G. (2011) Commensal microbiota and myelin autoantigen cooperate to trigger autoimmune demyelination. Nature 479(7374):538–41. Available at: https://doi.org/10.1038/nature10554.Google Scholar
Bramer, W. M., Giustini, D. & Kramer, B. M. (2016) Comparing the coverage, recall and precision of searches for 120 systematic reviews in Embase, MEDLINE, and Google Scholar: A prospective study. Systematic Reviews 5:39. Available at: https://doi.org/10.1186/s13643-016-0215-7.Google Scholar
Catalogue of Bias Collaboration, Aronson, J. K., Bankhead, C. & Nunan, D. (2017a) Hot stuff bias. In: Catalogue of bias. University of Oxford. Available at: https://catalogofbias.org/biases/hot-stuff-bias/.Google Scholar
Catalogue of Bias Collaboration, Plüddemann, A., Banerjee, A. & O'Sullivan, J. (2017b) Positive results bias. In: Catalogue of bias. University of Oxford. Available at: https://www.catalogueofbiases.org/biases/positive-results-bias.Google Scholar
Catalogue of Bias Collaboration, Spencer, E. A., Brassey, J. & Heneghan, C. (2017c) One-sided reference bias. In: Catalogue of bias. University of Oxford. Available at: https://www.catalogofbias.org/biases/one-sided-reference-bias.Google Scholar
Cheek, J., Garnham, B. & Quan, J. (2006) What's in a number? Issues in providing evidence of impact and quality of research(ers). Qualitative Health Research 16(3):423–35. Available at: https://doi.org/10.1177/1049732305285701.Google Scholar
Dickersin, K., Scherer, R. & Lefebvre, C. (1994) Identifying relevant studies for systematic reviews. British Medical Journal 309(6964):1286–91. Available at: https://doi.org/10.1136/bmj.309.6964.1286.Google Scholar
Greenberg, S. A. (2009) How citation distortions create unfounded authority: Analysis of a citation network. British Journal of Medicine 339:b2680. Available at: https://doi.org/10.1136/bmj.b2680.Google Scholar
Hafeez, D. M., Jalal, S. & Khosa, F. (2019) Bibliometric analysis of manuscript characteristics that influence citations: A comparison of six major psychiatry journals. Journal of Psychiatric Research 108:90–94. Available at: https://doi.org/10.1016/j.jpsychires.2018.07.010.Google Scholar
Ioannidis, J. P. A. (2005a) Contradicted and initially stronger effects in highly cited clinical research. JAMA 294(2):218–28.Google Scholar
Ioannidis, J. P. A. (2005b) Why most published research findings are false. PLoS Medicine 2(8):e124. Available at: https://doi.org/10.1371/journal.pmed.0020124.Google Scholar
Ioannidis, J. P. A., Boyack, K. W., Small, H., Sorensen, A. A. & Klavans, R. (2014) Bibliometrics: Is your most cited work your best? Nature 514:561–62. Available at: https://doi.org/10.1038/514561a.Google Scholar
Ioannidis, J. P. A. & Khoury, M. J. (2014) Assessing value in biomedical research: The PQRST of appraisal and reward. JAMA 312(5):483–84. Available at: https://doi.org/10.1001/jama.2014.6932.Google Scholar
Ioannidis, J. P. A. & Panagiotou, O. A. (2011) Comparison of effect sizes associated with biomarkers reported in highly cited individual articles and in subsequent meta-analyses. JAMA 305(21):2200–10. Available at: https://doi.org/10.1001/jama.2011.713.Google Scholar
Kulkarni, A. V., Aziz, B., Shams, I. & Busse, J. W. (2009) Comparisons of citations in Web of Science, Scopus, and Google Scholar for articles published in general medical journals. JAMA 302(10):1092–96. Available at: https://doi.org/10.1001/jama.2009.1307.Google Scholar
Lefebvre, C., Manheimer, E. & Glanville, J. (2011) Searching for studies. In: Cochrane handbook for systematic reviews of interventions version 5.1.0 (updated March 2011), ed. Higgins, J. P. T. & Green, S.. The Cochrane Collaboration. Available at: http://www.handbook.cochrane.org.Google Scholar
Mackinnon, S., Drozdowska, B. A., Hamilton, M., Noel-Storr, A. H., McShane, R. & Quinn, T. (2018) Are methodological quality and completeness of reporting associated with citation-based measures of publication impact? A secondary analysis of a systematic review of dementia biomarker studies. BMJ Open 8:e020331. Available at: https://doi.org/10.1136/bmjopen-2017-020331.Google Scholar
Rathbone, J., Carter, M., Hoffmann, T. & Glasziou, P. (2016) A comparison of the performance of seven key bibliographic databases in identifying all relevant systematic reviews of interventions for hypertension. Systematic Reviews 5:27. Available at: https://doi.org/10.1186/s13643-016-0197-5.Google Scholar
Seglen, P. O. (1998) Citation rates and journal impact factors are not suitable for evaluation of research. Acta Orthopaedica Scandinavica 69(3):224–29. Available at: https://doi.org/10.3109/17453679809000920.Google Scholar
Tajika, A., Ogawa, Y., Takeshima, N., Hayasaka, Y. & Furukawa, T. A. (2015) Replication and contradiction of highly cited research papers in psychiatry: 10-Year follow-up. British Journal of Psychiatry 207:357–62. Available at: https://doi.org/10.1192/bjp.bp.113.143701.Google Scholar
Tatsioni, A., Bonitsis, N. G. & Ioannidis, J. P. A. (2007) Persistence of contradicted claims in the literature. JAMA 298(21):2517–26. Available at: https://doi.org/10.1001/jama.298.21.2517.Google Scholar
Wieland, S. & Dickersin, K. (2005) Selective exposure reporting and Medline indexing limited the search sensitivity for observational studies of the adverse effects of oral contraceptives. Journal of Clinical Epidemiology 58(6):560–67. Available at: https://doi.org/10.1016/j.jclinepi.2004.11.018.Google Scholar
Target article
Microbiota-gut-brain research: A critical analysis
Related commentaries (19)
A call for mapping the development of the microbiota-gut-brain axis during human infancy
Beyond a gut feeling: How the immune system impacts the effect of gut microbiota in neurodevelopment
Combining integrated systems-biology approaches with intervention-based experimental design provides a higher-resolution path forward for microbiome research
Increasing reproducibility and interpretability of microbiota-gut-brain studies on human neurocognition and intermediary microbial metabolites
Inter-individual variation shapes the human microbiome
Microbiota-gut-brain research: A plea for an interdisciplinary approach and standardization
Neuropeptide-like signaling in the microbiota-gut-brain axis
Neurotropic enteroviruses co-opt “fair-weather-friend” commensal gut microbiota to drive host infection and central nervous system disturbances
Nourishing the gut microbiota: The potential of prebiotics in microbiota-gut-brain axis research
On the potential distortions of highly cited papers in emerging research fields: A critical appraisal
Practical guidelines for gut microbiome analysis in microbiota-gut-brain axis research
Putting microbiota-gut-brain research in a systemic developmental context: Focus on breast milk
Scientific claims are constitutive of common sense about health
Stress and microbiota: Between biology and psychology
The contribution of microbiology to neuroscience: More complex than it seems?
The parent-offspring microbiome and neurobehavioral development
Why a developmental cognitive neuroscience approach may be key for future-proofing microbiota-gut-brain research
Why don't probiotics work?
Why microbes, not microbiomes, are better causal explanations in gut-brain research
Author response
Causal clarity and deeper dimensions in microbiota-gut-brain research