Skip to main content Accessibility help

A tutorial for analyzing human reaction times: How to filter data, manage missing values, and choose a statistical model



This tutorial for the statistical processing of reaction times collected through a repeated-measure design is addressed to researchers in psychology. It aims at making explicit some important methodological issues, at orienting researchers to the existing solutions, and at providing them some evaluation tools for choosing the most robust and precise way to analyze their data. The methodological issues we tackle concern data filtering, missing values management, and statistical modeling (F1, F2, F1 + F2, quasi-F, mixed-effects models with hierarchical, or with crossed factors). For each issue, references and remedy suggestions are given. In addition, modeling techniques are compared on real data and a benchmark is given for estimating the precision and robustness of each technique.


Corresponding author

ADDRESS FOR CORRESPONDENCE Olivier Renaud, Methodology and Data Analysis, Department of Psychology, University of Geneva, 40 Boulevard du Pont d'Arve, 1211 Geneva 5, Switzerland. E-mail:


Hide All
Abelson, R., Rosenthal, R., Aiken, L., Appelbaum, M., Boodoo, G., Kenny, D., et al. (1996). Initial report—Task force on statistical inference. Washington, DC: American Psychological Association.
Baayen, R. H. (2008). Analyzing linguistic data. A practical introduction to statistics using R. Cambridge: Cambridge University Press.
Box, G. E. P. (1979). Robustness in the strategy of scientific model building. In Wilkerson, L. A. (Ed.), Robustness in statistics (pp. 201236). New York: Academic Press.
Clark, H. H. (1973). The language-as-fixed-effect fallacy: A critique of language statistics in psychological research. Journal of Verbal Learning and Verbal Behavior, 12, 335359.
Clark, H. H. (1976). Discussion of Wike and Church's comments. Journal of Verbal Learning and Verbal Behavior, 15, 257266.
Courvoisier, D. S., & Renaud, O. (2010). Robust analysis of the central tendency, simple and multiple regression and ANOVA: A step by step tutorial. International Journal of Psychological Research, 3, 78–87.
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, 39 (B), 138.
Fagot, D., Dirk, J., Ghisletta, P., and de Ribaupierre, A. (2009). Adults’ versus children's performance on the stroop task: Insights from ex-Gaussian analysis. Swiss Journal of Psychology, 68, 1724.
Forster, K. I., & Dickinson, R. G. (1976). More on the language-as-fixed-effect fallacy: Monte Carlo estimates of error rates for F1, F2, F′ and minF. Journal of Verbal Learning and Verbal Behavior, 15, 135142.
Goldstein, H. (2003). Multilevel statistical models (3rd ed.). London: Hodder Arnold.
Goldstein, H., Rasbash, J., Plewis, I., Draper, D., Yang, M., et al. (1998). A user's guide to MLwiN. London: Institute of Education.
Hox, J. (2002). Multilevel analysis: Techniques and applications. London: Erlbaum.
Kirk, R. E. (1968). Experimental design: Procedures for the behavioral sciences. Belmont, CA: Wadsworth.
Little, R. J. A., & Rubin, D. B. (1987). Statistical analysis with missing data. New York: Wiley.
Maronna, R., Martin, D., & Yohai, V. (2006). Robust statistics: Theory and methods. New York: Wiley.
Miller, R. G. (1997). Beyond ANOVA: Basics of applied statistics. Boca Raton, FL: Chapman & Hall.
Mitchell, M. L., & Jolley, J. M. (2007). Research design explained (6th ed.). Belmont, CA: Wadsworth Publishing.
R Development Core Team. (2010). R: A language and environment for statistical computing, robustbase package. Vienna, Austria: R Foundation for Statistical Computing.
Raaijmakers, J. G., Schrijnemakers, J. M., & Gremmen, F. (1999). How to deal with “the language-as-fixed-effect fallacy”: common misconceptions and alternative solutions. Journal of Memory and Language, 41, 416426.
Rasbash, J. (1992). Efficient computational procedures for the estimation of parameters in multilevel models based on iterative generalised least squares. Computational Statistics and Data Analysis, 13, 6371.
Rasbash, J., Steele, F., Browne, W., & Prosser, B. (2005). A user's guide to MLwiN version 2.0. Bristol: University of Bristol, Centre for Multilevel Modelling.
Ratcliff, R. (1993). Methods for dealing with reaction time outliers. Psychological Bulletin, 114, 510532.
Renaud, O., & Ghisletta, P. (2009). F1 + F2, F′, and multilevel model tests for experimental designs with two crossed random effects. Unpublished manuscript.
Rubin, D. B. (1987). Multiple imputation for nonresponse in surveys. New York: Wiley.
SAS Publishing. (2000). SAS/IML user's guide, version 8 (Vols. 1 and 2). Cary, NC: SAS Institute.
Satterthwaite, F. E. (1946). An approximation distribution of estimates of variance components. Biometrics, 2, 110114.
Schafer, J. L., & Graham, J. W. (2002). Missing data: Our view of the state of the art. Psychological Methods, 7, 147177.
Snijders, T., & Bosker, R. (1999). Multilevel Analysis: An introduction to basic and advanced multilevel modelling. London: Sage.
TIBCO Software Inc. (2008). TIBCO Spotfire S+ 8.1 Robust library user's guide. Palo Alto, CA: Author.
Ulrich, R., & Miller, J. L. (1994). Effects of truncation on reaction time analysis. Journal of Experimental Psychology: General, 123, 3480.
Wike, E. L., & Church, D. J. (1976). Comments on Clark's “The language-as-fixed-effect fallacy.” Journal of Verbal Learning and Verbal Behavior, 15, 249255.
Wilcox, R. R. (2005). Introduction to robust estimation and hypothesis testing. Reading, MA: Academic Press.
Winer, B. J., Brown, D. R., & Michels, K. M. (1991). Statistical principles in experimental design (3rd ed.). New York: McGraw–Hill.


Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed