WHAT DO WE (REALLY) KNOW ABOUT THE PAST?

Political scientists who use historical analysis first need to recognize how uncertain our knowledge about the past often is. For many historical periods and contexts that political scientists have begun to work with as part of the ongoing historical turn (e.g., the Early Middle Ages in Europe or Ancient China), the only firsthand written sources are a small number of miscellaneous documents such as law-codes, annals, and charters—which, at best, present partial information about the societies in question. Most information often comes from either archeology, which is difficult to interpret, or later secondhand narrative sources, which clearly—and often intentionally—misrepresent the past (Feng Reference Feng2008; Loewe and Shaughnessy Reference Loewe, Shaughnessy, Loewe and Shaughnessy1999; Wickham Reference Wickham2005).

To illustrate the problems that this creates, I begin with historian Chris Wickham’s (Reference Wickham2009, 12–18) discussion about the use of these secondhand narrative sources—in this case, from Early Medieval Europe. Wickham grants that many of these sources may be completely fictitious: examples include Gregory of Tours’s History of the Franks and Paul the Deacon’s History of the Langobards (see also Wickham Reference Wickham2005, 116–17). Moreover, even if they are only partly fictitious, we have little against which to check the information. However, Wickham assures us, this does not mean that these are not good sources for understanding the society for which they claim to write the history. Their moralizing observations thus are anchored in “recognizable experience.” At a minimum, therefore, these sources tell us about “the sort of things that could happen” (Wickham Reference Wickham2009, 14) in the world of the early Franks or the early Lombards. This insight guides how Wickham (Reference Wickham2005; Reference Wickham2009) uses this narrative evidence: not as a source of actual historical facts but rather as a source of what type of potential actions and reasoning characterized these societies.

The few and scattered firsthand sources are associated with other problems. What we often have are copies of copies, which makes it difficult to tell which part of the documents are original and which parts are later (i.e., secondhand) additions (Halsall Reference Halsall and Fourace2005, 61). Moreover, legal texts—one of the few relatively abundant sources of firsthand written material in many societies—often are better guides to the ambitions or values of elites than to the actual state of society (Wickham Reference Wickham2005, 95–6).

Three points follow from this. First, political scientists will not be able to eliminate trained historians as gatekeepers to the past—at least if they are dealing with premodern historical contexts. Not only does accessing the sources normally require specialist skills (e.g., a command of medieval Latin or training in locating relevant text in archives), the written material also cannot be taken at face value but rather must be interpreted by someone who understands the biases of different types of sources and who can place the interpretation within a particular historical context.

Second, political scientists must factor in the uncertainty of any historical interpretation. Here, the third historical source mentioned previously—archeology—comes into play. The interpretation of archeological findings obviously also requires expert skills and often involves creative leaps in the form of qualified guesses. However, another issue concerns the dynamic nature of our archeological knowledge. Wickham (Reference Wickham2005, 1) justified his magisterial attempt to revisit the Early Middle Ages in Europe with the fact that during a couple of decades, “[w]hat we can say about its archeology has multiplied tenfold—in some countries, a hundredfold.” This means that historical interpretations that factor in archeological findings quickly become outdated. Wickham (Reference Wickham2005, 10) makes clear that his own attempt to synthesize our knowledge about the Early Middle Ages—with the bulk of the evidence collected in 1997–2000—will first become dated in this respect. Therefore, political scientists who enlist older work of historians—including work that may be only a few decades old—must probe whether these interpretations still pass muster among historians or whether new archeological knowledge has made them obsolete.

Third, two additional issues further complicate matters. First, political scientists face the problem that there often are competing historical interpretations based on different perspectives on history. This means that they must become acquainted with the wider historiography to understand the different positions on a particular subject (Lustick Reference Lustick1996, 613). Second, among these competing historical interpretations, political scientists will be prone to emphasize those that confirm their theoretical arguments at the expense of those that do not accord well with them. This problem concerns confirmation bias in historical research (Lustick Reference Lustick1996, 608–10; Møller and Skaaning Reference Møller and Skaaning2018, 3–4), which is simply a more particular aspect of the general tendency that people find what they are looking for. This further underscores why we need to check the validity of the evidentiary record in historical analysis.

HOW TO CHECK THE READING OF THE WORK OF HISTORIANS?

That political scientists often treat history in a nonchalant or even superficial way is hardly a novel claim (e.g., Goldthorpe Reference Goldthorpe1991; Kreuzer Reference Kreuzer2010; Lustick Reference Lustick1996; Møller and Skaaning Reference Møller and Skaaning2018). Yet, based on my experience as both an author submitting papers with historical analysis and a reviewer of such papers (often with access to other reviewers’ comments), the problem seems to be widely ignored by the profession.

It might appear that the solution is simple. Returning to my initial analogy of the methodologist, we surely need to enlist at least one trained historian as reviewer to determine whether the reading of the work of historians passes muster. However, this solution has unfortunate side effects. First, as any political scientist who has submitted a historical analysis to a trained historian for comments knows, historians use different criteria for good research. Whereas political scientists normally try to generalize, compare, and present parsimonious explanations, historians focus on the specific and “gleefully proliferate variables all the time” (Gaddis Reference Gaddis2002, 58). Gaddis (2002, 56–58; see also 64, 88–91) presents this as the difference between the reductionist models of social scientists and the historians’ ecological approach. Even if historians were instructed that a paper should be reviewed as a product of political science, they almost certainly would be too critical of attempts to use historical material to generalize or make huge comparisons across space and/or time as well as the concomitant lack of attention to detail. Second, as previously mentioned, there are different schools within history, and representatives from one school (e.g., historians working in the Marxist tradition) would be overly critical regarding interpretations based on the work of other schools (e.g., historians working in the Annales tradition or political historians) (Hexter Reference Hexter and Hexter1979, 61–145; see also Lustick Reference Lustick1996, 615–6; Møller and Skaaning Reference Møller and Skaaning2018, 2–3). Editors of political science journals would be unlikely to factor in these differences between historical perspectives when reading reviews.

I believe the problem calls for another solution, which comes in two parts. First, and perhaps most important, editors and reviewers must be conscious of the fact that the use of historians’ work—whether for qualitative or quantitative purposes—cannot be taken simply at face value but instead needs to be questioned like any other type of data enlisted by political scientists. Second, we need a list of simple criteria against which historical work by political scientists can be assessed at minimal cost.

FOUR CRITERIA

I am not proposing a checklist that papers will either pass or fail. Rather, it should be perceived as a quickly applied set of criteria that can help reviewers—who are providing a community service—and editors identify whether historical claims seem tenuous. The metaphor is a warning signal, not a red light. If there is a warning, it often would make sense to raise this in reviews and, if the decision is to revise and resubmit, to ask authors to better substantiate and/or defend their reading of the work of historians, if necessary, in online appendices. I propose the following four criteria:

• • Are evidence claims about historical facts mainly based on work by historians rather than by social scientists?

• • Are evidence claims about historical facts mainly based on relatively new rather than relatively old historical sources?

• • Is there discussion or at least awareness of different historical interpretations and an attempt to adjudicate between these?

• • Does the author use page references for specific claims about historical facts?

These four criteria are not exhaustive; however, they are easy to apply and should flag blatant cases in which authors need to buttress their historical claims by invoking more relevant and/or unbiased historical data. The first three criteria are simpler versions of the more demanding criteria presented by Lustick (Reference Lustick1996, 615–6) and Møller and Skaaning (Reference Møller and Skaaning2018, 6–13). Their criteria cannot be used in the present form because they presuppose precisely what editors and reviewers normally lack—namely, a detailed knowledge of the historiography of a particular subject.

I previously clarified why we need the second and third criteria. The fourth—specific page numbers when referencing historical material—is particularly important when the claims that are invoked are not the central arguments or findings of the source but rather peripheral to it—or perhaps even made implicitly. In this case, the lack of page numbers makes it extremely cumbersome for readers to check these claims—or to read about them for their own use. Moreover, it is an affront to the historians who are invoked for evidence, considering that historians normally cite the particular pages for their evidence claims (Trachtenberg Reference Trachtenberg, Büthe and Jacobs2015, 13).

The first criterion—basing evidence claims mainly on work by historians rather than by social scientists—is a way to diminish the risk of confirmation bias by steering clear of a theoretically polluted selection or even reading of the evidence that historians initially produced (Møller and Skaaning Reference Møller and Skaaning2018, 8–11). This criterion is not meant to imply that the work of historians is unbiased; it also is affected by implicit theories, and historians often use social science concepts or theories that have long since been abandoned elsewhere. However, all else equal, work by historians has not to the same extent preselected evidence to provide support for one central theoretical claim (Gaddis Reference Gaddis2002, 56–58), which means that it is likely to be less polluted by the theoretical vantage point. Moreover, this criterion can be seen as a prudent way to check whether historical claims first encountered by reading other social scientists can be corroborated based on work by historians, including more recent work.

To illustrate how the four criteria can be applied, I use examples from two recent public-choice articles on the “Rise of Europe”: Salter (Reference Salter2015) and Salter and Young (Reference Salter and Young2018).^{Footnote 1} Salter (Reference Salter2015, 725) offers “a novel interpretation of the development of Western political institutions.” His claim is that when political authority is bundled with economic property rights, the result is a “shareholder state,” which he sees as responsible for the development of the European “nonpredatory governance” (Salter Reference Salter2015, 726). The main aim of his article is to present this theoretical argument. But Salter (Reference Salter2015, 728–9) makes clear that it is based on empirical underpinnings—that is, something resembling the “shareholder state” existed in Medieval Western Europe.

On what type of historical evidence does he base this claim? Salter (Reference Salter2015, 729) references the following authorities: “Anderson (Reference Anderson1991); Baechler (Reference Baechler1975); Benson (Reference Benson1990); Berman (Reference Berman1983); Raico (Reference Raico and Boettke1994); and Stark (Reference Stark2011, chaps. 14–6).” Only one of these scholars is a general historian (i.e., Baechler Reference Baechler1975); another is a legal historian (i.e., Berman Reference Berman1983). Both works are clearly dated and, throughout Salter’s article, there is no attempt to provide page references for the historical claims being made—this despite the fact that the main point of these sources is surely not that medieval monarchies resembled shareholder states. Neither is there any discussion about differences within historiography or even among the social scientists who are referred to for evidence about the medieval context. In a sense, Salter seems to take for granted that the medieval political order in fact did resemble his ideal type in which claims to political power and revenue derived from property rights.

Salter and Young (Reference Salter and Young2018) focus on the ways in which medieval representative institutions contributed to the Western European traditions of limited government. They ask why “the establishment of those traditions were uneven across countries” (2018, 173). To answer this, they revisit and elaborate on the German historian Otto Hintze’s (Reference Hintze and Hintze1962 [1930], 173) seminal argument “that two-chamber representative assemblies were more effective at resisting absolutist tendencies than three-chamber assemblies.” When presenting Hintze’s claim, Salter and Young (Reference Salter and Young2018, 178, fn. 14) use political scientist Thomas Ertman’s (Reference Ertman1997, 20–22) version of it.^{Footnote 2} When they describe the functioning of two- and three-chamber assemblies, respectively, they mainly refer to outdated historical evidence, including Bloch Reference Bloch1968a [1939], pp. 145–162; Vinogradoff Reference Vinogradoff, Gwatkin, Whitney, Tanner and Previté-Orton1968 [1922], p. 180; Lord Reference Lord1930, p. 37; Myers Reference Myers1975, p. 64 (fn. 20). Although Salter and Young should be praised for using page numbers, it seems fair to say that there is no attempt to probe the historical underpinnings of Hintze’s argument based on recent work by historians. In general, Salter and Young (Reference Salter and Young2018, 174) “take as historical givens the two types of assembly structures.”

In both cases, the question is why, during the review process, were the authors not tasked to provide firmer evidence in favor of these sweeping historical claims about the medieval context? Does the shareholder-state description of Medieval Europe and Hintze’s old claim about the institutional differences between the two categories of representative institutions, respectively, hold sway among historians and, if so, is it also corroborated by the most recent evidence? After reading the papers, we have almost no clue about the validity of the evidentiary record. As the historian J. H. Hexter (Reference Hexter and Hexter1979, 243–48) once critiqued the work of one of his colleagues, what we are left with is “an attractive hypothesis treading water, as it were, waiting for the solid ground of proof to be shoved under it. And that is not good enough.”