Skip to main content Accessibility help
×
Home

PEER-REVIEWED JOURNAL EDITORS’ VIEWS ON REAL-WORLD EVIDENCE

  • Elisabeth M. Oehrlein (a1), Jennifer S. Graff (a2), Eleanor M. Perfetto (a3), C. Daniel Mullins (a4), Robert W. Dubois (a2), Chinenye Anyanwu (a4) and Eberechukwu Onukwugha (a4)...

Abstract

Objectives: Peer-review publication is a critical step to the translation and dissemination of research results into clinical practice guidelines, health technology assessment (HTA) and payment policies, and clinical care. The objective of this study was to examine current views of journal editors regarding: (i) The value of real-world evidence (RWE) and how it compares with other types of studies; (ii) Education and/or resources journal editors provide to their peer reviewers or perceive as needed for authors, reviewers, and editors related to RWE.

Methods: Journal editors’ views on the value of RWE and editorial procedures for RWE manuscripts were obtained through telephone interviews, a survey, and in-person, roundtable discussion.

Results: In total, seventy-nine journals were approached, resulting in fifteen telephone interviews, seventeen survey responses and eight roundtable participants. RWE was considered valuable by all interviewed editors (n = 15). Characteristics of high-quality RWE manuscripts included: novelty/relevance, rigorous methodology, and alignment of data to research question. Editors experience challenges finding peer reviewers; however, these challenges persist across all study designs. Journals generally do not provide guidance, assistance, or training for reviewers, including for RWE studies. Health policy/health services research (HSR) editors were more likely than specialty or general medicine editors to participate in this study, potentially indicating that HSR researchers are more comfortable/interested in RWE.

Conclusions: Editors report favorable views of RWE studies provided studies examine important questions and are methodologically rigorous. Improving peer-review processes across all study designs, has the potential to improve the evidence base for decision making, including HTA.

  • View HTML
    • Send article to Kindle

      To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

      Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

      Find out more about the Kindle Personal Document Service.

      PEER-REVIEWED JOURNAL EDITORS’ VIEWS ON REAL-WORLD EVIDENCE
      Available formats
      ×

      Send article to Dropbox

      To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

      PEER-REVIEWED JOURNAL EDITORS’ VIEWS ON REAL-WORLD EVIDENCE
      Available formats
      ×

      Send article to Google Drive

      To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

      PEER-REVIEWED JOURNAL EDITORS’ VIEWS ON REAL-WORLD EVIDENCE
      Available formats
      ×

Copyright

This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.

References

Hide All
1. Sherman, RE, Anderson, SA, Dal Pan, GJ, et al. Real-world evidence — What is it and what can it tell us? N Engl J Med. 2016;375:22932297.
2. White, R. Building trust in real-world evidence and comparative effectiveness research: The need for transparency. J Comp Eff Res. 2017; 6:57.
3. Garrison, LP Jr, PJ, Neumann, Erickson, P, Marshall, D, Mullins, CD. Using real-world data for coverage and payment decisions: The ISPOR Real-World Data Task Force report. Value Health. 2007;10:326335.
4. Collins, FS, Varmus, H. A new initiative on precision medicine. N Engl J Med. 2015;372:793795.
5. Health Policy Brief: The FDA's Sentinel Initiative. June 4, 2015. [Internet]. http://www.healthaffairs.org/healthpolicybriefs/brief.php?brief_id=139. (accessed June 1, 2016).
6. PCORnet, the National Patient-Centered Clinical Research Network. October 19, 2017. [Internet]. http://www.pcornet.org/. (accessed November 13, 2017).
7. Gibson, TB, Ehrlich, ED, Graff, J, et al. Real-world impact of comparative effectiveness research findings on clinical practice. Am J Manag Care. 2014;20:e208–e220.
8. Ioannidis, J, Haidich, A, Pappa, M, et al. Comparison of evidence of treatment effects in randomized and nonrandomized studies. JAMA. 2001;286:821829.
9. Concato, J, Shah, N, Horwitz, RI. Randomized, controlled trials, observational studies, and the hierarchy of research designs. N Engl J Med. 2000;342:18871892.
10. MacLehose, RR, Reeves, BC, Harvey, IM, Sheldon, TA, Russell, IT, Black, AM. A systematic review of comparisons of effect sizes derived from randomised and non-randomised studies. Health Technol Assess. 2000;4:1154.
11. Anglemyer, A, Horvath, HT, Bero, L. Healthcare outcomes assessed with observational study designs compared with those assessed in randomized trials. Cochrane Database Syst Rev. 2014;4:MR000034.
12. Starks, H, Diehr, P, Curtis, J. The challenge of selection bias and confounding in palliative care research. Journal of Palliative Medicine. 2009;12:181187.
13. Greenfield, S, Kaplan, SH. Building useful evidence: Changing the clinical research paradigm to account for comparative effectiveness research. J Comp Eff Res. 2012;1:263270.
14. Villas Boas, PJ, Spagnuolo, RS, Kamegasawa, A, et al. Systematic reviews showed insufficient evidence for clinical practice in 2004: What about in 2011? The next appeal for the evidence-based medicine age. J Eval Clin Pract. 2013;19:633637.
15. Wager, E, Williams, Project, P Overcome failure to Publish nEgative fiNdings Consortium. “Hardly worth the effort”? Medical journals' policies and their editors' and publishers' views on trial registration and publication bias: Quantitative and qualitative study. BMJ. 2013;347:f5248.
16. Cals, JW, Mallen, CD, Glynn, LG, Kotz, D. Should authors submit previous peer-review reports when submitting research papers? Views of general medical journal editors. Ann Fam Med. 2013;11:179181.
17. Hing, CB, Higgs, D, Hooper, L, Donell, ST, Song, F. A survey of orthopaedic journal editors determining the criteria of manuscript selection for publication. J Orthop Surg Res. 2011;6:19.
18. Pouwels, KB, Widyakusuma, NN, Groenwold, RH, Hak, E. Quality of reporting of confounding remained suboptimal after the STROBE guideline. J Clin Epidemiol. 2016;69:217224.
19. Price, D, Bateman, ED, Chisholm, A, et al. Complementing the randomized controlled trial evidence base. Evolution not revolution. Ann Am Thorac Surg. 2014;11:S92S98.
20. Pain, E. Your data, warts and all. October 4, 2013. [Internet]. http://www.sciencemag.org/careers/2013/10/your-data-warts-and-all#.Uk9ffEYHu3c.twitter. (accessed April 20, 2016).
21. Editors. The registration of observational studies–When metaphors go bad. Epidemiology. 2010;21:607609.
22. Kreis, J, Panteli, D, Busse, R. How health technology assessment agencies address the issue of unpublished data. Int J Technol Assess Health Care. 2014;30:3443.
23. Sabharwal, RK, Graff, JS, Holve, E, Dubois, RW. Developing evidence that is fit for purpose: A framework for payer and research dialogue. Am J Manag Care. 2015;21:e545–e551.
24. Mullins, CD, Abdulhalim, AM, Lavallee, DC. Continuous patient engagement in comparative effectiveness research. JAMA. 2012;307:15871588.
25. Perfetto, EM, Burke, L, Oehrlein, EM, Epstein, RS. Patient-focused drug development: A new direction for collaboration. Med Care. 2015;53:917.
26. Schroter, S, Black, N, Evans, S, Godlee, F, Osorio, L, Smith, R. What errors do peer reviewers detect, and does training improve their ability to detect them? J R Soc Med. 2008;101:507514.
27. Haug, CJ. Peer-review fraud — Hacking the scientific publication process. N Engl J Med. 2015;373:23932395.
28. Sorokowski, P, Kulczycki, E, Sorokowska, A, Pisanski, K. Predatory journals recruit fake editor. Nature. 2017;543:481483.
29. EASE Guidelines for Authors and Translators of Scientific Articles to be Published in English. Acta Inform Med. 2014;22:210217.
30. Perfetto, EM, Anyanwu, C, Pickering, MK, Zaghab, RW, Graff, JS, Eichelberger, B. Got CER? Educating pharmacists for practice in the future: New tools for new challenges. J Manag Care Spec Pharm. 2016;22:609616.
31. MacMahon, B, Weiss, NS. Is there a dark phase of this STROBE? Epidemiology. 2007;18:791.
32. Shattell, MM, Chinn, P, Thomas, SP, Cowling, WR III. Authors' and editors' perspectives on peer review quality in three scholarly nursing journals. J Nurs Scholarsh. 2010;42:5865.
33. Weber, EJ, Katz, PP, Waeckerle, JF, Callaham, ML. Author perception of peer review: Impact of review quality and acceptance on satisfaction. JAMA. 2002;287:27902793.
34. Academy of Managed Care Pharmacy. The AMCP Format for Formulary Submissions Version 4.0. April 21, 2016. [Internet]. http://www.amcp.org/FormatV4/. (accessed November 13, 2017).
35. Weissman, JS, Westrich, K, Hargraves, JL, et al. Translating comparative effectiveness research into Medicaid payment policy: Views from medical and pharmacy directors. J Comp Eff Res. 2015;4: 7988.
36. Hurwitz, JT, Brown, M, Graff, JS, Peters, L, Malone, DC. Is real-world evidence used in P&T monographs and therapeutic class reviews? J Manag Care Spec Pharm. 2017;23:613620.
37. Rangarao Peck, S, Kleiner, H, Graff, JS, Lustig, A, Dubois, RW, Wallace, P. Are clinical practice guidelines being informed by real-world data? BMC Health Serv Res. In press.
38. Schnipper, LE, Davidson, NE, Wollins, DS, et al. American Society of Clinical Oncology Statement: A conceptual framework to assess the value of cancer treatment options. J Clin Oncol. 2015;33:25632577.
39. Makady, A, Ham, RT, de Boer, A, et al. Policies for use of real-world data in health technology assessment (HTA): A comparative study of six HTA agencies. Value Health. 2017;20:520532.

Keywords

Type Description Title
WORD
Supplementary materials

Oehrlein et al. supplementary material
Oehrlein et al. supplementary material 1

 Word (41 KB)
41 KB
WORD
Supplementary materials

Oehrlein et al. supplementary material
Oehrlein et al. supplementary material 2

 Word (60 KB)
60 KB
WORD
Supplementary materials

Oehrlein et al. supplementary material
Oehrlein et al. supplementary material 3

 Word (29 KB)
29 KB

PEER-REVIEWED JOURNAL EDITORS’ VIEWS ON REAL-WORLD EVIDENCE

  • Elisabeth M. Oehrlein (a1), Jennifer S. Graff (a2), Eleanor M. Perfetto (a3), C. Daniel Mullins (a4), Robert W. Dubois (a2), Chinenye Anyanwu (a4) and Eberechukwu Onukwugha (a4)...

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed