Manuscript Outcomes

The summary statistics presented above suggest minor overall differentiation in the rates of withdrawal and acceptance by manuscript type—and raise notable concerns for multiple women author teams. In this section, we test these differences for statistical significance in a regression context. Table 3 reports six logistic regression models.^{Footnote 5} In the first three, the dependent variable is equal to 1 if the manuscript was sent out for review (not withdrawn). In the remaining models, the sample includes only manuscripts that were sent out for review. The dependent variable is equal to 1 if the manuscript was accepted, and is 0 otherwise. The first four independent variables are indicator variables for each of the manuscript author-types. The Solo Man manuscripts are used as the primary reference category, while the third and sixth models use Multiple Men manuscripts as the reference. Several models also include the rank variable measured by the number of years between PhD (for the most experienced author) and manuscript submission. If no author with available data has a PhD, the rank is equal to 0. We report model coefficients in odds and p-values in parentheses.

Table 3 Logit Estimates in Odds Ratio, 2007–2017

Note: *p<0.1; **p<0.05; ***p<0.01

The choice of regressions reported relies on the assumption that the best comparison to make in order to assess bias against women in the review process is to compare (1) manuscript outcomes for single-authored works for men versus single-authored women and (2) multiple-authored works by teams of men versus teams of women. We discuss the results beyond these comparisons, but these comparisons are the primary benchmark we use to test for bias.

In terms of addressing the question of potential bias against women in the review process, the most important results to highlight are the coefficients in the first row of table 3 for models 1, 2, 4, and 5 and the third row for models 3 and 6. Solo Woman manuscripts are more likely to be sent out for review than Solo Man manuscripts. They are slightly less likely to be accepted; this result is not significant and disappears in model 5.^{Footnote 6} When we compare the outcomes for teams of all women versus teams of all men (models 3 and 6), we find that the odds of teams of all women being sent out for review are two times the odds that manuscripts by teams of men will be sent out for review. In terms of acceptance rates, manuscripts by all women are accepted at a substantially lower rate than manuscripts for teams of men, once they have been sent out for review; this result, however, is not statistically significant. There is no evidence of bias against women in either stage of the review process, according to these tests.

When considering manuscript author-type, the results in table 3 further suggest that Solo Woman, Multiple Women, Multiple Men, and Multiple Mixed are more frequently sent out for review relative to Solo Man manuscripts. Manuscripts by multiple women are most likely to be sent out for review. These results do not change substantially when we control for the rank of the most experienced author.

Once manuscripts have been sent out for review, any differences in acceptance rates between different manuscript types and Solo Man manuscripts are not statistically significant. In terms of magnitude, manuscripts by multiple women are substantially less likely to be accepted than Solo Man manuscripts, but this result is also not significant. Multiple Mixed Manuscripts are slightly more likely to be accepted than Solo Man, but again this result is not significant. Controlling for rank does not change the results substantially.^{Footnote 7}

These results further suggest that both Solo Man and Solo Woman manuscripts are less likely to be sent out for review, while manuscripts by teams of all women or of mixed gender are more likely to be sent for review than teams of men. Again, at the reviewer stage, none of the differences in acceptance rates among reviewed manuscripts are significant. Mixed teams fare slightly better than teams of men in terms of acceptance, but the difference is not significant.

Reviewer Recommendations

We now examine the recommendations made by individual (anonymized) reviewers to examine if these recommendations systematically vary depending on either the author-type of the manuscript or the gender of the reviewer. Importantly, reviewers undertake reviews without learning the name or gender of manuscript author(s). To preview our findings, there is no statistical evidence that reviewers recommend rejections more frequently for single-authored manuscripts by women versus men. Likewise, reviewers do not appear more likely to recommend a rejection for multi-authored manuscripts by all women relative to manuscripts by teams of all men. Reviewers are slightly less likely to recommend a rejection for teams with multiple authors of either mixed gender or all men compared with single authored manuscripts by men. Reviewers are slightly more likely to reject Solo Man and Solo Woman manuscripts (compared with teams of multiple men). The gender of the reviewer does not appear to have any significant effect on the recommendation. This result holds when we interact reviewer gender with author-type.

Before providing a more detailed account of the regression results we discuss the data used in this section and provide several descriptive tables. This section examines reviewer decisions. Table 4 provides background summary information for this data, as well as summary information on individual authors (who might submit one or more manuscripts in our dataset). There are 2,996 total unique manuscripts for which we have author gender information and a final decision outcome. There are 3,199 unique authors who collectively submitted these manuscripts. Some manuscripts are submitted by multiple authors and some authors submit multiple manuscripts. Just under 30% of all unique authors are women. Because many manuscripts are withdrawn in the first stage of the editorial process, the pool of manuscripts with associated reviewer information is a subset of all manuscripts. In our data, there are 2,124 unique manuscripts with final decisions that have gender information for authors and at least one of the reviewers. We have gender information for 1,841 unique reviewers associated with these manuscripts. Of the pool of reviewers, 28.2% are women. Women comprise a comparable share of authors and reviewers in this dataset.^{Footnote 8}

Table 4 Manuscripts, Author, Reviewer Breakdown, 2007–2017

Next, we examine the distribution of reviewer decisions by reviewer gender. The unit of analysis in table 5 and in the regression analyses below is each reviewer decision on each manuscript. Thus, observations are unique at the manuscript-reviewer level. We examine the recommendations that reviewers make on the first submission only; we do not examine the decisions on revision submissions, although we have done this elsewhere and the results are consistent with what we report here.^{Footnote 9} Reviewers have four possible recommendations they can make: (1) Publishable essentially as is, (2) Publishable subject to revisions, (3) Revise and Resubmit, and (4) Reject. Table 5 summarizes the recommendations by the reviewer gender. In general, men and women provide similar shares of each recommendation type. Men tend to recommend “Publish essentially as is” and “Publish subject to revisions” slightly more frequently than women; women recommend “Revise & Resubmit” and “Reject” slightly more frequently.

Table 5 Reviewer Decision Breakdown by Reviewer Gender, 2007–2017

To test these differences as well as examine possible variation in reviewer recommendation by manuscript author-type, we estimate several logistic regression models.^{Footnote 10} In each of the models, we use a binary outcome variable that is equal to 1 if the reviewer recommends that the manuscript be rejected and 0 if any of the other three possible recommendations are made that would keep the manuscript “alive.” The manuscript author-types are the same as those used in the earlier regressions. We include an indicator variable that is equal to 1 if the reviewer is a woman. In the first two models, we examine the independent effect of reviewer gender, manuscript author-type, and rank of the most experienced author on the reviewer’s recommendation. In models 3 through 5, we also interact the gender of the reviewer with the author-type and rank. Models 1 through 4 use Solo Man manuscripts as the reference category, while model 5 provides the full specification using manuscripts authored by teams of men as the reference category.

Table 6 indicates that women reviewers do not recommend rejections more frequently than reviewers who are men. This result holds regardless of the author-type of the manuscript.^{Footnote 11} In terms of differences in recommendations based on the author-type of the manuscript, we find that Solo Woman manuscripts are slightly less likely to receive a reject recommendation than Solo Man manuscripts, but the difference is small and not significant (coefficient on Solo Woman with Solo Man as reference category is 0.939, in model 1). Reviewers are also less likely to recommend the rejection of multi-authored manuscripts by teams of all men (0.846) or mixed-gender teams (0.792) than Solo Man manuscripts; these differences are statistically significant. Manuscripts by teams of women are slightly more likely (1.002) to receive a reject recommendation relative to Multiple Men, according to model 5, although this finding is not statistically significant. In general, the results of reviewer decisions corroborate the findings associated with manuscript outcomes.

Table 6 Reviewer Recommendation Logit Estimates in Odds Ratio, 2007–2017

Note: *p<0.1; **p<0.05; ***p<0.01

Interesting to note, we find that the rank, measured in years since PhD of the most experienced author of a manuscript, is positively associated with a rejection recommendation. That is, the more experienced the highest-ranking author, the more likely it is that a reviewer will recommend that the manuscript is rejected; the magnitude of this coefficient is quite small. For completeness, in models 3 through 5 we additionally check for any interaction effects between manuscript type and reviewer’s gender. None of these interaction effects are significant.

Time Trends

We first examine the time trends in the number of submissions by different author-types. Figure 1 segments the annual submission data into three categories: (1) manuscripts written by one or multiple men, (2) manuscripts written by one or multiple women, and (3) manuscripts written jointly by at least one man and at least one woman.^{Footnote 12} As the chart demonstrates, and consistent with much of the data discussed above, the clear majority of manuscripts fall into the first category—with the second and third categories submitting at a significantly lower rate. The share of manuscripts in each category is relatively stable over time. There is a slight decline in manuscripts submitted by men and a slight increase in the manuscripts submitted by mixed teams.

Figure 1 Percentage of All Submissions by Manuscript Author Type

In figure 2, we break this data down further into the five author-types featured in the rest of the analysis. The graph helps visualize the problem of the extremely low share of manuscripts that are submitted by multiple women.

Figure 2 Breakdown of Submitted Manuscripts by Author Type

Breakdown by Rank

Next, we examine submission data based on the rank of the most experienced authors of a given manuscript. In table 7, we segment rank into three categories (based on available information): (1) the most experienced author does not have a PhD or its equivalent (or earned their PhD in the same calendar year that the manuscript was submitted), (2) the most experienced author earned their PhD between one and seven years prior to the date the manuscript was submitted, and (3) the most experienced author earned their PhD more than seven years prior to submitting the manuscript.^{Footnote 13} While there is much valuable information within this chart, we highlight just a few characteristics of the data. First, submissions for both Solo Man and Solo Woman manuscripts peak during the earlier years of authors’ post-PhD career. Second, the majority of coauthored pieces, regardless of gender, have at least one senior contributor. Third, at each career stage fewer women submit solo manuscripts than men. At each of the three career stages, men submit 76.1, 67.8 and 77.5% of solo manuscripts, respectively. As a point of reference, in 2014 men represented 60% of graduate students, 61% of assistant professors, 67% of associate professors and 77% of full professors (Teele and Thelen Reference Teele and Thelen2017, 436). Thus, the biggest discrepancy in our 2007–2017 data between representation in the field and in submission rates occurs at the graduate student level. Fourth, graduate students who are women are much less likely to submit manuscripts both as solo authors and especially as teams. Of the 71 manuscripts submitted by teams of only graduate students, only 7% were from teams of all women, while teams of all men represent nearly 51% of these submissions.

Table 7 Submissions by Author Type and Rank, 2007–2017

The data just presented provide a breakdown of gender and experience at the level of manuscript submissions. We can also consider trends at the level of author-manuscript, where each manuscript corresponds to several possible observations, one for each author involved in writing a given piece. In the first panel of figure 3, along the vertical axis, we plot the absolute number of observations in four possible author statuses (solo man, male author with coauthors of either gender, solo woman, and female author with coauthors of either gender). Along the horizontal axis, the counts are disaggregated by the number of years since the author earned their PhD. In the second panel, for each experience level, the share of observations for each author category is plotted.

Figure 3 Author Submission by Experience (Count—Left, Share—Right)

Note: For purposes of chart clarity, we only report authors with 40 or fewer years of experience.

Trends in the submission data based on author experience indicate that the number and share of solo female authors drops precipitously with experience. The absolute number of women submitting with coauthors also declines, but not until experience exceeds six years with PhD. The share of women submitting with coauthors does not decline, it even increases slightly and unevenly among authors later in their careers. The trends look somewhat different for men. Solo-authorship is common among men early in their career. The absolute number of men submitting solo-authored work declines monotonically (first panel). The share of men submitting alone (versus all other categories) decreases between 0 and 15 years of experience but does not vary systematically after that. Moreover, with the exception of several anomalous experience levels, in general, as men stop submitting solo-authored work, they seem to compensate by submitting coauthored work.

We should further probe why fewer women submit to World Politics in the middle and later stages of their academic careers. Since the pool of female scholars shrinks with years of experience and the share of women in political science has increased over time, broader disciplinary trends, including cohort effects, might provide further insight into these patterns (Teele and Thelen Reference Teele and Thelen2017).^{Footnote 14} It is worth remembering, however, that generational trends in women’s rank and roles in professional political science activities are likely endogenous to gendered dynamics in the submission and publication process. It might also be that more senior women are shifting their publication strategies by submitting articles to a different set of journals and/or focusing on publishing books over articles. Arguably harder to track, research productivity and manuscript submission patterns might also reflect the different and disproportionate demands that women face relative to men in the workplace and at home, as sociological studies might lead one to hypothesize.^{Footnote 15} Why coauthorship rates by senior women do not increase with seniority (in contrast to male peers), is a question worthy of further analysis, as noted in the conclusion.

Patterns of Cross-Rank Coauthorship

Finally, we examine the patterns of coauthorship across ranks. In table 8 we consider teams of authors with various gender-rank combinations for all manuscripts (with complete gender and rank information for all authors) between 2007 and 2017. In the first and second columns, we examine teams of one or more students (defined as 0 years since PhD at time of submission) and one or more authors who earned a PhD at least one year prior to the date of submission. Each row indicates different gender structures in cross-rank teams. A “W” or “M” indicates that these manuscripts have at least one and possibly more women (W); or at least one and possibly more men in the associated rank position (M). In the third and fourth columns, we repeat the exercise, but using different rank cut-points. Here, junior is defined as anyone with seven years or less since PhD (including students) at the time of submission and senior is anyone with more than seven years since PhD. The second to last row in the table details a count for all manuscripts in which there is no cross-rank coauthorship; of note, cross-rank coauthorship constitutes only 11.1% and 15.7% of all manuscripts, respectively, across the two sets of teams. Thus, the trends discussed in this section are only a small part of the overall submission story. Given space constraints, the following text exclusively discusses the student-PhD coauthored teams, but we refer the readers to table 8 for further data regarding junior–senior coauthor teams.

Table 8 Cross-Rank Coauthorship, 2007–2017

Note: W indicates female and M male.

At the graduate level, it is worth highlighting that more male students than female students coauthor with scholars who already have their PhD (168 teams with one or more male graduate students versus 87 teams with female graduate students, not including teams with male and female students). Thus, teams with female graduate students are 34.1% of these cross-rank coauthorships in World Politics submission data, but women are about 40% of graduate students.^{Footnote 16} Our data indicate that men and women with PhDs may show preference for coauthoring with students of their same gender. Of teams where the more senior scholar(s) is a woman, 45.8% are with one or more female students, while 54.2% are with one or more male students (again excluding teams with male and female students). If we assume for the sake of this comparative exercise that there is only one student and one advanced scholar on each team, then senior women tend to over-select female graduate students for coauthorship since the latter are only 40% of graduate students.^{Footnote 17} Likewise, of teams where the more senior scholar is a man, 30.3% are with one or more female students, while 69.7% are with male students; in these teams, male graduate students tend to be over-represented (as men are estimated to compose 60% of political science graduate students) while female graduate students are under-represented.

If World Politics submissions trends are consistent with other journals, the cross-rank coauthorship patterns may suggest several reasons that female students (and junior scholars) participate in fewer cross-rank submissions than their male counterparts: male mentors might not equally seek them out as coauthors; female mentors (who slightly favor them as coauthors) might submit fewer articles than their male peers; and lack of gender parity on faculty might amplify these uneven mentoring/coauthorship patterns.^{Footnote 18} In light of these data and other research on mentorship, further investigations into mentorship, professionalization and networks may be one of several important avenues for redressing overall underrepresentation of women in journal submissions and publications.^{Footnote 19}