As social and behavioral scientists, it is of fundamental importance to understand the factors that drive the behaviors that we measure. Careful design is thus required to minimize the influence of extraneous factors. Yet, we often overlook one major class of such extraneous factors – those related to us, the experimenters. Experimenter effects can potentially arise at every step in the research process – from the selection of hypotheses, to interacting with research participants in ways that might alter their behavior, to biases in data interpretation. While such experimenter-driven effects often occur without notice, and without ill intent, they nonetheless threaten the replicability and generalizability of research. In this chapter, we discuss when and how such effects arise, preventative measures that can be taken to reduce their influence, and methods for accounting for such effects, when appropriate.

Working memory (WM) training explores whether and how repeated practice on working memory tasks might generalize to a variety of outcome measures. Although this field of research is part of the growing literature in cognitive sciences, it has spawned contentious debates. The controversies are largely driven by inconsistent findings and commercial interests, and as a result, numerous meta-analyses and systematic reviews have focused on the validity of WM training. Similarly, there is an inconsistency in the conclusions drawn by these meta-analyses; while there seems to be an agreement about the generalization to proximal cognitive measures; there is a discrepancy in the interpretation of any translational outcomes (e.g., behavioral, clinical, and academic). In this chapter, we review the collection of meta-analyses with a particular focus on children diagnosed with ADHD and other developmental disabilities, and recommend that the field should focus on improving our understanding of the mechanistic and effectiveness properties of WM training, which might result in the development of valuable alternative and/or supplemental approaches, when traditional interventions might fall short, especially for individuals typically underrepresented and underserved.