Although men and women earn bachelor’s degrees in political science at approximately equal rates, only 40% of PhDs in political science are awarded to women, and only 35% of assistant professors and 22% of full professors are women (Nelson Reference Nelson, Nelson and Cheng2017). Women are less likely to publish in top journals or to be cited or promoted (Hesli, Lee, and Mitchell Reference Hesli, Lee and Mitchell2012; Mitchell and Hesli Reference Mitchell and Hesli2013; Teele and Thelen Reference Teele and Thelen2017). Dozens of solutions have been proposed to mitigate the gender gap (e.g., APSA Workshop on the Advancement of Women 2005; Monroe et al. Reference Monroe, Choi, Howell, Lampros-Monroe, Trejo and Perez2014). For example, the report of the APSA Committee on the Status of Women in the Profession (2016) provided more than 70 recommendations that address the full gamut of potential barriers and suggest a wide variety of policy ideas. For example, the report advocates for the creation of diversity postdoc positions, expanding childcare programs, and language standards for tenure documents or review letters, among many others. However, the 2016 task force report cited only one published research article demonstrating the effectiveness of a recommended intervention (i.e., a study of mentoring by Blau et al. Reference Blau, Currie, Croson and Ginther2010) (APSA Committee on the Status of Women in the Profession 2016). Although much important work has documented barriers to women’s advancement and proposed solutions have proliferated, there has been comparatively little systematic analysis of the effect of interventions.
We extend this work by reviewing research that evaluates the effectiveness of interventions. We discuss all published systematic evaluations of interventions in political science and related fields and present four findings. First, little research exists on this question. The majority of proposed recommendations have never been tested. This implies the need to prioritize such research in the future. Second, there is good evidence for the utility of women’s mentoring programs. Third, we find suggestive evidence in favor of changes in hiring practices, resilience training, department service dashboards, and student evaluations. However, our fourth finding is that many other interventions yield negligible, null, or even negative effects on women’s advancement. Given the failure of several widely promoted programs, we conclude that political scientists should prioritize systematic evaluation before interventions are broadly implemented.
…little research exists on this question. The majority of proposed recommendations have never been tested.
SCOPE AND METHOD OF REVIEW
We systematically searched for research that evaluates interventions to improve women’s advancement in political science and related fields. This included any study that implemented a specific program or change intended to improve women’s outcomes in academic political science and evaluated its effect. We included as our “evidence” only those studies published in journals or books since 2008 that evaluated the effectiveness of an intervention designed to improve outcomes for graduate students and faculty (we omitted studies of undergraduate recruitment or retention). Within those parameters, we intended for this review to be as exhaustive as possible and not biased by researcher priorities. We supplemented the evidence with references to research that does not meet the criteria but provides useful information (theoretical or contextual), clearly noting when we did so.
Evaluation is defined as comparing the intervention group to a group that did not receive it.Footnote 1 Ideally, this would involve random assignment, which allows for a causal attribution of observed differences to the tested intervention. However, due to the low incidence of randomization in this area and because randomization is not always feasible, we included any study that used an untreated comparison group, explicitly noting the presence or absence of randomization. (More information about study identification is in the online appendix.)
We found only three studies that tested interventions in political science and evaluated their effect relative to a comparison group: Barnes and Beaulieau (Reference Barnes and Beaulieu2017); Peterson et al. (Reference Peterson, Biederman, Andersen, Ditonto and Roe2019); and Unkovic, Sen, and Quinn (Reference Unkovic, Sen and Quinn2016). Because there are so few, we expanded the scope of research to include other disciplines, giving more weight to social science fields and cross-disciplinary and university-wide interventions. When we used evidence beyond political science and related social sciences, we focused on interventions that address aspects of women’s psychology, publishing, and university structure that we expected to function similarly across disciplines. A comprehensive and cross-disciplinary discussion of the sources of academic gender gaps is beyond the scope of this review. However, for each intervention, we briefly explain why we think it addresses a particular gendered process that poses a barrier to women’s advancement in political science—regardless of the field in which the intervention was tested. To the extent that interventions tested in other disciplines are intended to mitigate similar underlying barriers, that evidence can be informative in political science. (A more complete discussion of cross-disciplinary relevance is in the online appendix.)
Encouragement and Resilience
One well-documented explanation for the gender gap in women’s performance in academia and beyond is that women are more likely than men to lack confidence in their abilities (Karpowitz and Mendelberg Reference Karpowitz and Mendelberg2014, ch. 2). Accordingly, some interventions aim to improve women’s outcomes by increasing the confidence and resilience of women in academia.
Unkovic, Sen, and Quinn (Reference Unkovic, Sen and Quinn2016) sent an email invitation to apply for the annual summer meetings of the Society for Political Methodology (PolMeth) to a randomly selected half of graduate students attending top-50 political science programs. The invitations increased applications from men by 2 percentage points and from women by 3 percentage points. However, invited women who applied were less likely to be accepted than both women in the control group and men in the treatment group, primarily because they were less likely to submit the required faculty reference letters. Although the encouragement design increased applications, the null effect on acceptances highlights the pitfalls of interventions that rely on women to overcome barriers to securing their own supporters. It also points to the importance of including relevant behavioral measures in evaluations.
In a more successful approach, Bekki et al. (Reference Bekki, Smith, Bernstein and Harrison2013) reported the positive impact of online resilience training (i.e., CareerWISE). Compared with the delayed-access control group, female STEM graduate students who were randomly assigned to spend at least 5 hours on the website reported substantially higher levels of problem-solving knowledge and moderately higher resilience and coping efficacy. However, there were null effects on four additional outcomes. In addition, outcomes were self-reported and measured within two weeks of accessing the training site; therefore, the longevity and behavioral impacts are unknown as well as the applicability beyond STEM.
Descriptive Representation among Decision Makers
Increasing the number of women in academic decision-making positions has been widely recommended on the assumption that they will exhibit less gender bias. However, the evidence suggests otherwise. In a randomized experiment, Moss-Racusin et al. (Reference Moss-Racusin, Dovidio, Brescoll, Graham and Handelsman2012) asked 127 science faculty to evaluate resumés of identical applicants for a laboratory-manager position with randomly varied male and female names. Both male and female faculty rated the male as more competent and hirable, offering him a higher starting wage and more mentoring.
In Spain and Italy, randomly composed committees conducted nationwide reviews of candidates for academic promotion (in all academic fields). Women candidates fared no better—and worse in some situations—with additional women on the committee (Bagues, Sylos-Labini, and Zinovyeva Reference Bagues, Sylos-Labini and Zinovyeva2017). A caveat is that the female percentage was low in the majority of committees (i.e., less than 40%). Research finds that women make a difference only if they are given power—for example, when they are a majority or have a veto (Karpowitz and Mendelberg Reference Karpowitz and Mendelberg2014). It stands to reason, then, that a small increase in women’s representation on committees will fail.
Consistent with the evidence of biased evaluations of women, some scholars have speculated that women’s lower representation in top disciplinary journals is related to gendered biases in the peer-review process (Teele and Thelen Reference Teele and Thelen2017). However, the evidence on the use of double-blind review as a mechanism to narrow the publication gap is mixed. One study in ecology found that women’s proportion of publications increased by 7.9% after the journal changed from single- to double-blind review, with no observed change in a comparable single-blind journal (Budden et al. Reference Budden, Tregenza, Aarssen, Koricheva, Leimu and Lortie2008; Roberts and Verhoef Reference Roberts and Verhoef2016). However, more recent observational analysis found no difference between single- and double-blind journals, including in those same journals (Cox and Mongomerie 2019). In political science, the gender gap in authorship in journals is less a result of biased editorial processes (Brown and Samuels Reference Brown and Samuels2018) than of gender gaps in the submission pool (Brown et al. Reference Brown, Horiuchi, Htun and Samuels2020). This provides further evidence that changes to the gender composition or process of reviews are unlikely to improve women’s advancement in publication and promotion.
Hiring Process
One leak in the political science pipeline that is common across many disciplines is the loss of women in the transition from graduate school to their first tenure-track job (APSA Committee on the Status of Women in the Profession 2016). Changes to the hiring process may increase the number of women interviewed, offered jobs, and hired. However, these changes may fail to retain women over the long term.
When the University of California–Irvine (UC–I) implemented university-wide “equity advisers” (i.e., senior faculty assigned to monitor gender equity throughout the hiring process), the female proportion of faculty increased by 2 points more than the growth rate at other UC campuses. However, women subsequently left the university at a higher rate than other campuses (Stepan-Norris and Kerrissey Reference Stepan-Norris and Kerrissey2015). Equity advisers may be insufficient to improve retention.Footnote 2
Montana State University’s science departments randomly assigned a diversity program to half of 23 faculty searches. Women composed 40% of short-listed and interviewed candidates in treated searches, compared to 20% in control searches in the same year. Treated searches made six times more offers to women (Smith et al. Reference Smith, Handley, Zale, Rushing and Potvin2015). A voluntary hiring program at the University of Wisconsin–Madison also demonstrated positive correlations between program participation and women’s hiring of similar magnitude (Sheridan et al. Reference Sheridan, Fine, Pribbenow, Handelsman and Carnes2010). Thus, hiring interventions show some promise albeit with two important caveats. First, the numbers of faculty searches are small, implying the need for additional rigorous evaluation and replication. Second, retention is rarely measured but, when it is, evidence suggests a null effect.
Gender-Neutral Clock-Stopping
Gender-neutral tenure clock-stopping (GNCS) policies stop the tenure clock, typically for one year, for male and female faculty members when they have a child. These policies attempt to mitigate the “chronological time crunch” wherein childrearing overlaps with high job demands (APSA Committee on the Status of Women in the Profession 2016).
Antecol, Bedard, and Stearns (Reference Antecol, Bedard and Stearns2018) tracked the tenure and publication outcomes for every newly hired assistant professor at a top-50 economics department from 1980 to 2005 and correlated them with the year that the department adopted a GNCS policy. Contrary to expectations, women hired after-versus-before GNCS policy adoption were 22 percentage points less likely to achieve tenure at the hiring institution and there was no change in top-five journal publication rates. For men, GNCS policy adoption increased top-five publications and raised tenure rates by 19 percentage points. That is, GNCS backfired for women but helped men, thereby exacerbating the gender gap.
Similarly, using survey data, Feeney, Bernal, and Bowman (Reference Feeney, Bernal and Bowman2014) found that GNCS has no relationship with STEM faculty women’s self-reported journal publications. More generally, they found that relatively generous family-leave policies have only a weak positive correlation with women’s publication rates. Other interventions in this study with no observed positive correlation with women’s outcomes included university “Status of Women” reports, official spousal hiring policies, on-site childcare, and flexible scheduling.
Teaching and Service Expectations
Women may face gender bias in student evaluations (Mitchell and Martin Reference Mitchell and Martin2018) and heavier service loads (Pyke Reference Pyke2011). Two studies reported successful interventions against these barriers.
In four large introductory biology or political science courses—two sections each, with one male and one female instructor—Peterson et al. (Reference Peterson, Biederman, Andersen, Ditonto and Roe2019) randomly assigned some students within each class to receive either a short statement in the evaluation instructions about the potential for bias or no statement. Women instructors received higher teaching evaluations (i.e., 0.3 to 0.5 point on a 5-point scale) from students in the treatment condition than in the control condition, with no effect on evaluations of male instructors. Social desirability could explain the results—and the effects may decay if students tune out similar alerts in other courses—but the results were promising enough to justify additional testing.
O’Meara et al. (Reference O’Meara, Jaeger, Misra, Lennartz and Kuvaeva2018) addressed the problem of unequal service. Thirty STEM departments were match-randomized to a four-part intervention: (1) a workshop on implicit bias, (2) a “dashboard” to collect and share transparent information about faculty service assignments, (3) a “Department Equity Action Plan” based on the dashboard information, and (4) an optional professional-development webinar series. Faculty in treated (versus untreated) departments reported a more equitable workload and were more likely to state that they were comfortable self-advocating. The treatment improved outcomes for men but more so for women. Although they lack behavioral or long-term assessment, the findings invite additional tests.
Diversity Training
Diversity training represents a low-cost means to change exclusionary practices by providing information about bias. Unfortunately, the evidence does not support this goal (Moss-Racusin et al. Reference Moss-Racusin, van der Toorn, Dovidio, Brescoll, Graham and Handelsman2014; Paluck and Green Reference Paluck and Green2009). Any positive effects of mandatory training last only a few days, and it often triggers resistance (Dobbin and Kalev Reference Dobbin and Kalev2016). Randomized trials found only small effects and on only a few self-reported attitudinal measures (e.g., Carnes et al. Reference Carnes, Devine, Manwell, Byars-Winston, Fine, Ford, Forscher, Isaac, Kaatz, Magua, Palta and Sheridan2015; Pietri et al. Reference Pietri, Hennes, Dovidio, Brescoll, Bailey, Moss-Racusin and Handelsman2018; Zawadzki et al. Reference Zawadzki, Shields, Danube and Swim2014).
Moreover, diversity training may backfire for female scientists. Those who were randomized to receive training (versus no training) had a lower sense of belonging in the sciences, a more negative affect, and greater social-identity threat levels (Pietri et al. Reference Pietri, Hennes, Dovidio, Brescoll, Bailey, Moss-Racusin and Handelsman2018). These effects were counteracted by presenting a role model or optimistic messaging; overall, however, diversity training does not work well enough to justify implementation or even much further testing.
Mentorship and Networking Opportunities
Mentoring and networking programs stand out for demonstrated substantial positive effects on important outcomes.
Mentoring and networking programs stand out for demonstrated substantial positive effects on important outcomes.
The American Economic Association hosts annual two-day mentoring workshops for women (i.e., CeMENT). Blau et al. (Reference Blau, Currie, Croson and Ginther2010) evaluated the CVs of women who applied and were randomly admitted or not. Five years later, attendees were 23 to 27 points more likely to have a federal grant or a top-tier publication and they had 2.4 more publications.
In political science, Barnes and Beaulieu (Reference Beaulieu, Boydstun, Brown, Dionne, Gillespie, Klar, Krupnikov, Michelson, Searles and Wolbrecht2017) reported a positive correlation between attending Visions in Methodology (VIM)—a recurring political methodology workshop for women—and various career outcomes. They surveyed workshop participants and a nonrandom comparison group of women and men from the same departments who had never attended. Women who attended VIM reported substantially more mentorship and peer support within and beyond their institution than comparison men or women; 0.3 and 0.7 more articles submitted than comparison men and women; 0.5 more invited talks than comparison men but not women; and more submissions to top-three journals than comparison women but no difference with men. Other women’s workshops in political science include Journeys in World Politics, New Research on Gender in Political Psychology (Bos and Schneider Reference Bos and Schneider2012), APSA’s Women of Color workshop, and International Studies Association’s Pay It Forward. Combined with the randomized evaluation of the CeMENT workshops, observational evidence from these conferences suggests that mentoring and networking can be effective and should be the focus of further evaluation.
Workshops—and mentoring programs generally—are a bundle of treatments. Many possible mechanisms could explain the positive results. Future research should replicate CeMENT’s design, including randomized assignment, long-term follow-up, and measures of important outcomes (e.g., publications and tenure). It also should identify the mechanisms so that subsequent efforts can efficiently implement only the effective elements. Potential mechanisms include developing a peer network (Barnes and Beaulieu Reference Barnes and Beaulieu2017), perhaps particularly composed of female peer mentors (Bostwick and Weinberg Reference Bostwick and Weinberg2018; Dennehy and Dasgupta Reference Dennehy and Dasgupta2017); receiving quality mentorship from senior faculty (Blau et al. Reference Blau, Currie, Croson and Ginther2010; Curtin, Malley, and Stewart Reference Curtin, Malley and Stewart2016); receiving useful feedback on work in progress (Barnes and Beaulieu Reference Barnes and Beaulieu2017); building confidence and providing encouragement (Curtin, Malley, and Stewart Reference Curtin, Malley and Stewart2016; Karpowitz and Mendelberg Reference Karpowitz and Mendelberg2014); developing psychological resilience (Bekki et al. Reference Bekki, Smith, Bernstein and Harrison2013); receiving information about professionalization (APSA Committee on the Status of Women in the Profession 2016; Curtin, Malley, and Stewart Reference Curtin, Malley and Stewart2016); having female role models (Carrell, Page, and West Reference Carrell, Page and West2010); and receiving support and empowerment from all-women environments (Karpowitz and Mendelberg Reference Karpowitz and Mendelberg2018).
Such efforts in political science are underway. In 2018, APSA sponsored a women’s networking and mentoring workshop in Washington, DC, featuring random assignment of applicants to attend or not. This will allow long-term causal evaluation of mentoring effects in political science (APSA Presidential Task Force on Women’s Advancement in the Profession 2019). Furthermore, recent case studies (without a comparison group) show potential for building on these mechanisms using virtual peer-mentoring and networking opportunities, such as writing groups (Cassese and Holman Reference Cassese and Holman2018) and Women Also Know Stuff (Beaulieu et al. Reference Beaulieu, Boydstun, Brown, Dionne, Gillespie, Klar, Krupnikov, Michelson, Searles and Wolbrecht2017).
CONCLUSIONS
In our review of interventions in political science and related fields, we found that many practices and programs have not been evaluated, and those that have been often turn out to be ineffective or even backfire. The many small, null, or negative effects underscore the need to evaluate interventions before they are recommended for implementation. Efforts should shift from compiling long lists of potential solutions to increasingly rigorous tests of those most likely to be implemented widely and to work effectively. Especially needed are studies that (1) conduct close comparisons with a matched and preferably randomized control group; (2) measure effects over the long term; (3) include the most important outcomes (i.e., publications, tenure, retention, and satisfaction with one’s workplace); (4) do not rely exclusively on self-reports; and (5) assess outcomes in both absolute terms and relative to men. As political scientists, we should apply the methodological rigor of our discipline to the important question of how to increase women’s advancement within the field.
As political scientists, we should apply the methodological rigor of our discipline to the important question of how to increase women’s advancement within the field.
That said, research provides some evidence for specific interventions. The most promising are mentoring and networking workshops for women. Other specific interventions show tentative evidence and therefore are worth further evaluation, including interventions in hiring practices, departmental service, teaching evaluations, and resilience training.
Supplementary Materials
To view supplementary material for this article, please visit http://dx.doi.org/10.1017/S1049096520000402.
