Following the publication of our original article (El Kurd and Hummel Reference El Kurd and Hummel2023), a reader brought to our attention an error in the experimental analysis that lowers the p-value of one intervention to p=.08. While evaluating the error and its cause, we found and corrected additional minor errors in the code and appendix. We detail those errors here.
Errors
The main error is in the experimental analysis. The original published analysis uses the raw outcome variable from the survey results. This outcome variable overstates variance in the response. In the online survey, it was possible during the first hours of deployment to respond on a scale of 1 to 100 instead of 1 to 10. We quickly changed the scale to match the question, which asked for respondents to rate their response from 1 to 10. The raw data contains a few responses over 10 from that error and thus the raw outcome variable overstates the possible variance. During the data cleaning process, we re-coded those responses onto a 1-10 scale and created a new variable, ReportMisconduct, where each point in the scale is equidistant from the others. Due to miscommunication between the author who cleaned the data and the author who did the experimental analysis, the latter used the raw variable instead of the transformed variable, leading to the error.
In El Kurd and Hummel (Reference El Kurd and Hummel2023), we reported that students who received the racial harassment prime were less likely to report harassment than students who received the neutral prime and that this difference was significant at p<0.01. This is not true. Using the transformed variable, students who received the racial harassment prime were 13% less likely to report harassment than students who received the neutral prime and this difference is significant at p<0.1 (p=.08). Students who received the sexual harassment prime were also less likely to report harassment than students who received the neutral prime using either version of the variable, but this result has not had a p-value below 0.1 in any analysis.
Users should use the ReportMisconduct variable to analyze the experiment. Because of the errors and confusion, we have removed the raw variable from the replication data and code.
While we were evaluating the source of the error, we realized that the experimental conditions were mislabeled during the data cleaning process. This is an error in the original data cleaning code. We have fixed the error and clearly labeled the conditions. This error was simply confusing and did not change any reported results or interpretations.
We also realized that the table of experimental results in the original online appendix reported results from an earlier analysis where we had included respondents outside of the sampling frame. We had dropped those respondents for all analyses reported in the original published manuscript. We have removed those respondents from the table in the appendix and updated the table with information from the sample as reported in the original published manuscript.
Finally, we found typos in the appendix that we have corrected, including some coefficients and standard errors that were incorrectly recorded in the appendix tables. These were human errors from copying a number into the wrong column. None of these errors resulted in changed signs, significance falling below conventional levels, or altered the results reported in the main text.
Acknowledgements
Many thanks to Dr. LJ Zigerell Jr. for bringing this problem to our attention and to the editors for comments and assistance. Updated data and code that corrects the issue are available on the PS Dataverse and an updated online appendix is available at: https://doi.org/10.7910/DVN/KN28MM
Open access
