Madole & Harden (M&H) propose that within-family genetic effects are analogous to randomized controlled trials (RCTs), and therefore can help identify potential genetic causes of psychological phenomena. In doing so, the authors repeat the frequent trope of RCTs as the “gold standard” (target article, sect. 1.1, para. 2) for establishing causal claims. Although it is clear that RCTs are optimally set up for such a task, using the language of “gold standard” has been criticized for obscuring the many limitations of the design (see Deaton & Cartwright [Reference Deaton and Cartwright2018] for a thorough treatment of the subject and Jones & Podolsky [Reference Jones and Podolsky2015] for a historical discussion). The uncritical use of RCTs as an analogue for genetic effects opens their arguments to many criticisms, ones that are already prominent in work on behavior genetics. Among others, these include insufficient attention to conceptualization (Nguyen, Syed, & McGue, Reference Nguyen, Syed and McGue2021) and measurement (Pelt, Schwabe, & Bartels, Reference Pelt, Schwabe and Bartels2022) of the target phenotype and lack of sample diversity when seeking to establish generalizable claims (Holden, Haughbrook, & Hart, Reference Holden, Haughbrook and Hart2022). By not incorporating these major concerns in their paper, M&H unfortunately compound, rather than resolve, these issues in their otherwise productive set of arguments.

The more general criticism levied toward both RCTs and behavior genetics is the over-emphasis on method over substance. M&H continue in this tradition, devoting careful attention to establishing and calibrating causal claims. This is clearly a useful and much-needed approach to thinking about causation, especially in psychology, but we must not overlook the quality of the substantive information about which we seek to make causal claims. Here, I highlight just one aspect of such quality, the racial/ethnic diversity of samples included in behavior genetic studies, and why such a consideration must be central to any effort to build generalizable causal knowledge.

It is a fact of the design that RCTs sacrifice external validity for the sake of internal validity, being high in efficacy, showing promising results in trials, but low in effectiveness, or lack of results when translated to real-life conditions (Flay et al., Reference Flay, Biglan, Boruch, Castro, Gottfredson, Kellam and Ji2005). RCTs have been further criticized by researchers in multicultural psychology and culturally adapted treatments for their lack of inclusion of racial/ethnic minorities and thus limited generalizability (Bernal & Scharrón-del-Río, Reference Bernal and Scharrón-del-Río2001; Castro, Barrera, & Holleran Steiker, Reference Castro, Barrera and Holleran Steiker2010; Whaley & Davis, Reference Whaley and Davis2007). Consistent with the mainstream view in psychology, the arguments made by M&H all assume a universal, replaceable person; it does not matter where the information comes from, so long as the proper modeling is applied (see also Yarkoni, Reference Yarkoni2022). But, of course, strong design features cannot overcome selection issues that can lead to misspecified models (Bradley et al., Reference Bradley, Kuriwaki, Isakov, Sejdinovic, Meng and Flaxman2021).

The research on lithium as a treatment for manic symptoms of bipolar disorder, highlighted by the authors, illustrates their lack of attention to sample diversity. The second-generation studies cited by M&H – those that seek to identify specific causal mechanisms beyond an average treatment effect – relied entirely on data from White men (Mertens et al., Reference Mertens, Wang, Kim, Yu, Pham, Yang and Yao2015; Santos et al., Reference Santos, Linker, Stern, Mendes, Shokhirev, Erikson and Gage2021; Stern et al., Reference Stern, Santos, Marchetto, Mendes, Rouleau, Biesmans and Gage2018). It may well be that the identified mechanisms are generalizable beyond this very narrow group, but it seems prudent to investigate this question prior to broadly claiming generalizable causal knowledge. Moreover, the authors frame this kind of second-generation investigation as addressing the problem of lack of generalizability beyond the first-generation average treatment effect, but it does so only with respect to individual differences, and not demographic/population heterogeneity.

An argument could be made to justify sample homogeneity in first-generation studies, when putative causal factors are initially identified, and explore generalizability as part of the second-generation process. This is related to the issue of “portability” of findings from genetic studies (or “effectiveness” with RCTs), which refers to the fact that average effects identified through first-generation behavior genetics studies may not generalize to new contexts or populations. Indeed, the impressive Lee et al. (Reference Lee, Wedow, Okbay, Kong, Maghzian, Zacher and Cesarini2018) study of educational attainment of 1.1 million individuals included only participants with European ancestry. When the researchers attempted to “port” the polygenic scores derived from the European-ancestry group to a sample of Black Americans, the 10.6% R ² attenuated by 85%, a result that they indicated was “typical of what has been reported in other studies” (p. 1115). M&H fail to mention the degree of this problem, let alone what the implications are for building generalizable causal knowledge. For example, the omnigenic model highlights the need for diversity in discovery samples to identify and separate both core and peripheral variants (Mathieson, Reference Mathieson2021; see also Wojcik et al., Reference Wojcik, Graff, Nishimura, Tao, Haessler, Gignoux and Carlson2019). Thus, diversity is central to first-generation studies.

Moreover, in discussions of portability, left unsaid is from whom to whom the results are being ported, which is nearly always from White/European samples to other ancestry or racial groups. Rarely do we seek to, for example, generalize results from African samples to other groups (Adetula, Forscher, Basnight-Brown, Azouaghe, & IJzerman, Reference Adetula, Forscher, Basnight-Brown, Azouaghe and IJzerman2022). This dynamic sets up a standard in which the White/European results serve as the basis for the first-generation causal knowledge, and any deviations from it are problems to be solved or, more often, swept under the rug. Such a perspective is consistent with the deficit model that has for decades dominated psychological research on diversity (Cauce, Coronado, & Watson, Reference Cauce, Coronado, Watson, Hernandez and Mareasa1998), and unreasonably constrains the context of discovery.

The issues mentioned here are reminiscent of the aphorism “garbage in, garbage out” in the context of meta-analyses. That is, no degree of sophisticated analyses will save your substantive conclusions if the studies included therein are weak or uninformative. To be fair, M&H clearly know this, but the issue is treated more in passing rather than as a central concern in their quest to build stronger causal knowledge, a quest which I greatly support. They use a catch phrase in similar structure to “garbage in, garbage out” when discussing causal reasoning, “no causes in, no causes out,” which, fittingly, pertains to the reasoning and not the quality of the information within it. Combining these two, I might say, “no diversity in, no causes at all.”