Burt casts doubt on the utility of polygenic scores (PGSs) in social science research. I applaud her attempt to avoid ad hominem attacks and straw man arguments. Although she largely succeeds at the former, I argue below that she sometimes fails at the latter. And although many of the limitations she describes are valid, her conclusion – that these represent fatal flaws in the use of PGSs in social science research – is not.
Burt begins and ends her paper by drawing an analogy between the excitement previously generated by the candidate gene approach and that surrounding the use of PGSs in social and behavioral genetics research. I do not believe this analogy holds. The candidate gene era laid bare the fallibility of the scientific process as currently practiced: It is likely that the many thousands of positive candidate gene findings on psychiatric and behavioral traits reported in the literature are predominated by false positives (Border et al., Reference Border, Johnson, Evans, Smolen, Berley, Sullivan and Keller2019). However, research findings on PGSs are very different. PGS findings are largely replicable and PGSs estimate true quantities. The issue at hand is how PGSs should be used and how their results should be interpreted. This is a much different and much more interesting place to be.
A core problem with Burt's critique of PGSs is that she misconstrues the state of thinking in the field, and then proceeds to argue against her own misconstruction. For example, what Burt calls “downward causation” refers to the context dependence of genetic associations, which, she argues, makes genetic associations for social behaviors “unavoidably confounded.” Yet no behavioral geneticist (I hope) believes that genetic effects exist in a vacuum, independent from any potential environmental context. Alleles that influence smoking may often have different effects depending on public policy and availability of tobacco, and alleles that influence skin pigmentation probably have different influences on vitamin D sufficiency in societies that differ in average sun exposure, because of clothing or climate. This does not make these genetic effects “artificial” – it simply means they are mediated by environmental factors – nor does it make them “unavoidably confounded” – mediation and confounding are conceptually distinct. Such mediation makes PGSs more, not less, interesting, and would reduce enthusiasm for studying PGSs only if one expects that “true” genetic effects should be invariant across context. (Whether genetic effects differ across extant environmental differences is, of course, an empirical question.)
Similarly, Burt points out that simplifying assumptions made in genome-wide association studies (GWASs) (e.g., that single-nucleotide polymorphisms [SNPs] have only additive effects) or in construction of some PGSs (e.g., that all SNPs are causal) are wrong, or that the approach (e.g., only estimating effects at common variants) ignores important information. She uses these observations to imply that these approaches are therefore naïve or produce untrustworthy results. However, models are not meant mirror reality – to be so would not only be impossible but would render them incomprehensible. Models intentionally simplify to be understandable and/or to allow parameter estimation. Contrary to Burt's black-or-white thinking on this, at issue is the degree to which results are biased and whether this bias matters with respect to the question being investigated. It is well understood that PGSs underestimate total trait heritability, mostly because of the finite sizes of GWAS discovery samples (Wray et al., Reference Wray, Yang, Hayes, Price, Goddard and Visscher2013). Depending on the question at hand, the underestimation may often be irrelevant (e.g., a hypothesis test of whether a depression PGS predictive ability is moderated by stressful life events; Colodro-Conde et al., Reference Colodro-Conde, Couvy-Duchesne, Zhu, Coventry, Byrne, Gordon and Martin2018) or be corrected for in the model (e.g., using structural equation modeling on PGSs within families to estimate parental influences; Balbona, Kim, & Keller, Reference Balbona, Kim and Keller2021). The imperfect predictive ability of PGSs has no necessary relationship to their utility.
More centrally, Burt argues that PGSs of social traits are likely to be biased because of indirect genetic effects (e.g., passive gene–environment correlation or assortative mating) or because of confounding with environmental differences (e.g., as a result of uncontrolled population stratification). This arguably may not matter much if the goal of the PGS is purely to predict (Plomin & von Stumm, Reference Plomin and von Stumm2022), but it certainly matters if the goal is explanation. Again, however, this is problematic only to the extent that (a) the confounding effects exist and have not been corrected (which will inevitably occur to some degree, depending on the trait and design), and (b) that the results are interpreted as being solely because of direct genetic influences. It should be noted that the issues Burt raises regarding the interpretation of PGS results apply equally to the interpretation of GWAS effect sizes, SNP-heritability, and SNP-correlation estimates. So should these approaches be used “sparingly” when studying social or behavioral outcomes, as Burt argues? I think not. Many of the interpretational issues Burt raises are real, but they are inherent to the topics of study. Understanding the causes of individual differences in, say, educational attainment is complicated business, and must involve a tangled interplay of genetic, social, and environmental factors, all mediated through multiple different channels, but this is cause for more and better investigation, not a reason to refrain from researching one of the potentially important factors (genetics) influencing educational attainment.
Burt has identified several core issues regarding the difficulty in interpreting molecular genetic estimates, but these are neither unique to PGSs nor to social/behavioral traits. How should the field move forward in light of these issues? In agreement with Burt, there should be greater care in interpreting and describing PGS results, for example, as the relationship between a trait and “PGS estimates” rather than “genetic propensity.” There is increasing awareness in the field that gene–environment correlations are real and can mislead if interpreted as being purely because of direct genetic effects – driven largely by findings from sociogenomics researchers (Abdellaoui, Dolan, Verweij, & Nivard, Reference Abdellaoui, Dolan, Verweij and Nivard2022; Berg et al., Reference Berg, Harpak, Sinnott-Armstrong, Joergensen, Mostafavi, Field and Coop2019; Howe et al., Reference Howe, Nivard, Morris, Hansen, Rasheed, Cho and Palviainen2022; Kong et al., Reference Kong, Thorleifsson, Frigge, Vilhjalmsson, Young, Thorgeirsson and Stefansson2018; Young et al., Reference Young, Frigge, Gudbjartsson, Thorleifsson, Bjornsdottir, Sulem and Kong2018). An alternative tack is to use new designs and/or data types that allow disambiguation of environmental and genetic effects. One obvious approach is to oversample close relatives in future collections of biobank style datasets. Such within-family estimates may not provide perfect estimates of direct genetic effects, but they do control for the vast majority of potentially confounding environmental influences (Howe et al., Reference Howe, Nivard, Morris, Hansen, Rasheed, Cho and Palviainen2022).
In summary, I have a much more optimistic view of the future of PGS research in social science than does Burt, even with its imperfections and challenges. The challenges make the topic all the more worthy of careful and innovative investigation.
Burt casts doubt on the utility of polygenic scores (PGSs) in social science research. I applaud her attempt to avoid ad hominem attacks and straw man arguments. Although she largely succeeds at the former, I argue below that she sometimes fails at the latter. And although many of the limitations she describes are valid, her conclusion – that these represent fatal flaws in the use of PGSs in social science research – is not.
Burt begins and ends her paper by drawing an analogy between the excitement previously generated by the candidate gene approach and that surrounding the use of PGSs in social and behavioral genetics research. I do not believe this analogy holds. The candidate gene era laid bare the fallibility of the scientific process as currently practiced: It is likely that the many thousands of positive candidate gene findings on psychiatric and behavioral traits reported in the literature are predominated by false positives (Border et al., Reference Border, Johnson, Evans, Smolen, Berley, Sullivan and Keller2019). However, research findings on PGSs are very different. PGS findings are largely replicable and PGSs estimate true quantities. The issue at hand is how PGSs should be used and how their results should be interpreted. This is a much different and much more interesting place to be.
A core problem with Burt's critique of PGSs is that she misconstrues the state of thinking in the field, and then proceeds to argue against her own misconstruction. For example, what Burt calls “downward causation” refers to the context dependence of genetic associations, which, she argues, makes genetic associations for social behaviors “unavoidably confounded.” Yet no behavioral geneticist (I hope) believes that genetic effects exist in a vacuum, independent from any potential environmental context. Alleles that influence smoking may often have different effects depending on public policy and availability of tobacco, and alleles that influence skin pigmentation probably have different influences on vitamin D sufficiency in societies that differ in average sun exposure, because of clothing or climate. This does not make these genetic effects “artificial” – it simply means they are mediated by environmental factors – nor does it make them “unavoidably confounded” – mediation and confounding are conceptually distinct. Such mediation makes PGSs more, not less, interesting, and would reduce enthusiasm for studying PGSs only if one expects that “true” genetic effects should be invariant across context. (Whether genetic effects differ across extant environmental differences is, of course, an empirical question.)
Similarly, Burt points out that simplifying assumptions made in genome-wide association studies (GWASs) (e.g., that single-nucleotide polymorphisms [SNPs] have only additive effects) or in construction of some PGSs (e.g., that all SNPs are causal) are wrong, or that the approach (e.g., only estimating effects at common variants) ignores important information. She uses these observations to imply that these approaches are therefore naïve or produce untrustworthy results. However, models are not meant mirror reality – to be so would not only be impossible but would render them incomprehensible. Models intentionally simplify to be understandable and/or to allow parameter estimation. Contrary to Burt's black-or-white thinking on this, at issue is the degree to which results are biased and whether this bias matters with respect to the question being investigated. It is well understood that PGSs underestimate total trait heritability, mostly because of the finite sizes of GWAS discovery samples (Wray et al., Reference Wray, Yang, Hayes, Price, Goddard and Visscher2013). Depending on the question at hand, the underestimation may often be irrelevant (e.g., a hypothesis test of whether a depression PGS predictive ability is moderated by stressful life events; Colodro-Conde et al., Reference Colodro-Conde, Couvy-Duchesne, Zhu, Coventry, Byrne, Gordon and Martin2018) or be corrected for in the model (e.g., using structural equation modeling on PGSs within families to estimate parental influences; Balbona, Kim, & Keller, Reference Balbona, Kim and Keller2021). The imperfect predictive ability of PGSs has no necessary relationship to their utility.
More centrally, Burt argues that PGSs of social traits are likely to be biased because of indirect genetic effects (e.g., passive gene–environment correlation or assortative mating) or because of confounding with environmental differences (e.g., as a result of uncontrolled population stratification). This arguably may not matter much if the goal of the PGS is purely to predict (Plomin & von Stumm, Reference Plomin and von Stumm2022), but it certainly matters if the goal is explanation. Again, however, this is problematic only to the extent that (a) the confounding effects exist and have not been corrected (which will inevitably occur to some degree, depending on the trait and design), and (b) that the results are interpreted as being solely because of direct genetic influences. It should be noted that the issues Burt raises regarding the interpretation of PGS results apply equally to the interpretation of GWAS effect sizes, SNP-heritability, and SNP-correlation estimates. So should these approaches be used “sparingly” when studying social or behavioral outcomes, as Burt argues? I think not. Many of the interpretational issues Burt raises are real, but they are inherent to the topics of study. Understanding the causes of individual differences in, say, educational attainment is complicated business, and must involve a tangled interplay of genetic, social, and environmental factors, all mediated through multiple different channels, but this is cause for more and better investigation, not a reason to refrain from researching one of the potentially important factors (genetics) influencing educational attainment.
Burt has identified several core issues regarding the difficulty in interpreting molecular genetic estimates, but these are neither unique to PGSs nor to social/behavioral traits. How should the field move forward in light of these issues? In agreement with Burt, there should be greater care in interpreting and describing PGS results, for example, as the relationship between a trait and “PGS estimates” rather than “genetic propensity.” There is increasing awareness in the field that gene–environment correlations are real and can mislead if interpreted as being purely because of direct genetic effects – driven largely by findings from sociogenomics researchers (Abdellaoui, Dolan, Verweij, & Nivard, Reference Abdellaoui, Dolan, Verweij and Nivard2022; Berg et al., Reference Berg, Harpak, Sinnott-Armstrong, Joergensen, Mostafavi, Field and Coop2019; Howe et al., Reference Howe, Nivard, Morris, Hansen, Rasheed, Cho and Palviainen2022; Kong et al., Reference Kong, Thorleifsson, Frigge, Vilhjalmsson, Young, Thorgeirsson and Stefansson2018; Young et al., Reference Young, Frigge, Gudbjartsson, Thorleifsson, Bjornsdottir, Sulem and Kong2018). An alternative tack is to use new designs and/or data types that allow disambiguation of environmental and genetic effects. One obvious approach is to oversample close relatives in future collections of biobank style datasets. Such within-family estimates may not provide perfect estimates of direct genetic effects, but they do control for the vast majority of potentially confounding environmental influences (Howe et al., Reference Howe, Nivard, Morris, Hansen, Rasheed, Cho and Palviainen2022).
In summary, I have a much more optimistic view of the future of PGS research in social science than does Burt, even with its imperfections and challenges. The challenges make the topic all the more worthy of careful and innovative investigation.
Financial support
M.C.K. was supported by NIH grants MH100141 and MH130448.
Competing interest
None.