Skip to main content Accessibility help

Reference Rot: An Emerging Threat to Transparency in Political Science

  • Aaron L. Gertler (a1) and John G. Bullock (a2)


Transparency of research is a large concern in political science, and the practice of publishing links to datasets and other online resources is one of the main methods by which political scientists promote transparency. But the method cannot work if the links don’t, and very often, they don’t. We show that most of the URLs ever published in the American Political Science Review no longer work as intended. The problem is severe in recent as well as in older articles; for example, more than one-fourth of links published in the APSR in 2013 were broken by the end of 2014. We conclude that “reference rot” limits the transparency and reproducibility of political science research. We also describe practices that scholars can adopt to combat the problem: when possible, they should archive data in trustworthy repositories, use links that incorporate persistent digital identifiers, and create archival versions of the webpages to which they link.



Hide All
Ainsworth, Scott G., Nelson, Michael L., and Sompel, Herbert Van de. 2015. “Only One Out of Five Archived Web Pages Existed as Presented.” Presented at the Annual Meeting of the ACM Conference on Hypertext and Social Media, Cyprus. (accessed April 30, 2016).
Altman, Micah and King, Gary. 2007. “A Proposed Standard for the Scholarly Citation of Quantitative Data.” D-Lib Magazine 13 (March-April).
Askitas, Nikos. 2010 June. “What Makes Persistent Identifiers Persistent?” German Data Forum. Manuscript. (accessed July 26, 2015). Archived at
Center for Research Libraries. 2006. “ICPSR Audit Report.” default/files/d6/attachments/pages/ICPSR_final.pdf (accessed August 27, 2015). Archived at
Dafoe, Allan. 2014. “Science Deserves Better: The Imperative to Share Complete Replication Files.” PS: Political Science & Politics 47 (1): 6066.
Data Access and Research Transparency. 2014. “Data Access and Research Transparency (DA-RT): A Joint Statement by Political Science Journal Editors.” (accessed August 27, 2015). Archived at
Data Preservation Alliance for the Social Sciences. 2014. “Memorandum of Understanding.” (accessed August 27, 2015). Archived at
Dimitrova, Daniela V. and Bugeja, Michael. 2007. “The Half-Life of Internet References Cited in Communication Journals.” New Media & Society 9 (5): 811–26.
Gerber, Alan and Malhotra, Neil. 2008. “Do Statistical Reporting Standards Affect What Is Published?” Quarterly Journal of Political Science 3 (October): 313–26.
Harvard Dataverse. 2015. “Harvard Dataverse Preservation Policy.” (accessed July 27, 2015). Archived at
International DOI Foundation. 2015. “DOI System and the Handle System.” http: // (accessed August 27, 2015). Archived at
Internet Archive. 2015. “Removing Documents from the Wayback Machine.” (accessed July 26, 2015). Archived at http://perma. cc/ZY3P-RX45.
King, Gary. 2007. “An Introduction to the Dataverse Network as an Infrastructure for Data Sharing.” Sociological Methods & Research 36 (2): 173–99.
Lepore, Jill. 2015. “The Cobweb.” The New Yorker, January 26, 3341.
Lupia, Arthur and Elman, Colin. 2014. “Openness in Political Science: Data Access and Research Transparency.” PS: Political Science & Politics 47 (1): 1924.
Monogan, James E III. 2015. “Research Preregistration in Political Science: The Case, Counterarguments, and a Response to Critiques.” PS: Political Science & Politics 48 (3): 425–29.
Van de Sompel, Herbert and Treloar, Andrew. 2014. “A Perspective on Archiving the Scholarly Web.” Proceedings of the 11th International Conference on Digital Preservation. (accessed August 27, 2015). Archived at
Wagner, Cassie, Gebremichael, Meseret D., Taylor, Mary K., and Soltys, Michael J.. 2009. “Disappearing Act: Decay of Uniform Resource Locators in Health Care Management Journals.” Journal of the Medical Library Association 97 (2): 122–30.
Zittrain, Jonathan, Albert, Kendra, and Lessig, Lawrence. 2014. “Perma: Scoping and Addressing the Problem of Link and Reference Rot in Legal Citations.” Harvard Law Review 127 (February): 176–99.
Type Description Title
Supplementary materials

Gertler and Bullock supplementary material
Online Appendix

 PDF (84 KB)
84 KB

Reference Rot: An Emerging Threat to Transparency in Political Science

  • Aaron L. Gertler (a1) and John G. Bullock (a2)


Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed