We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.
To save content items to your account,
please confirm that you agree to abide by our usage policies.
If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account.
Find out more about saving content to .
To save content items to your Kindle, first ensure no-reply@cambridge.org
is added to your Approved Personal Document E-mail List under your Personal Document Settings
on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part
of your Kindle email address below.
Find out more about saving to your Kindle.
Note you can select to save to either the @free.kindle.com or @kindle.com variations.
‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi.
‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.
This chapter focuses on causal inference in healthcare, emphasizing the need to identify causal relationships in data to answer important questions related to efficacy, mortality, productivity, and care delivery models. The authors discuss the limitations of randomized controlled trials due to ethical or pragmatic considerations and introduce quasi-experimental research designs as a scientifically coherent alternative. They divide these designs into two broad categories, independence-based designs and model-based designs, and explain the validity of assumptions necessary for each design. The chapter covers key concepts such as potential outcomes, selection bias, heterogeneous treatment effects bias, average treatment effect, average treatment effect for the treated and untreated, and local average treatment effect. Additionally, it discusses important quasi-experimental designs such as regression discontinuity, difference-in-differences, and synthetic controls. The chapter concludes by highlighting the importance of careful selection and application of these methods to estimate causal effects accurately and open the black box of healthcare.
People live complicated lives and, unlike laboratory scientists who can control all aspects of their experiments, epidemiologists have to work with that complexity. As a result, no epidemiological study can ever be perfect. Even an apparently straightforward survey of, say, alcohol consumption in a community, can be fraught with problems. Who should be included in the survey? How do you measure alcohol consumption reliably? All we can do when we conduct a study is aim to minimise error as far as possible, and then assess the practical effects of any unavoidable error. A critical aspect of epidemiology is, therefore, the ability to recognise potential sources of error and, more importantly, to assess the likely effects of any error, both in your own work and in the work of others. If we publish or use flawed or biased research we spread misinformation that could hinder decision-making, harm patients and adversely affect health policy. Future research may also be misdirected, delaying discoveries that can enhance public health.
Paternal exposures (and other non-maternal factors) around pregnancy could have important effects on offspring health. One challenge is that data on partners are usually from a subgroup of mothers with data, potentially introducing selection bias, limiting generalisability of findings. We aimed to investigate the potential for selection bias in studies using partner data.
We characterise availability of data on father/partner and mother health behaviours (smoking, alcohol, caffeine and physical activity) around pregnancy from three UK cohort studies: the Avon Longitudinal Study of Parents and Children (ALSPAC), Born in Bradford and the Millennium Cohort Study. We assess the extent of sample selection by comparing characteristics of families where fathers/partners do and do not participate. Using the association of parental smoking during pregnancy and child birthweight as an example, we perform simulations to investigate the extent to which missing father/partner data may induce bias in analyses conducted only in families with participating fathers/partners.
In all cohorts, father/partner data were less detailed and collected at fewer timepoints than mothers. Partners with a lower socio-economic position were less likely to participate. In simulations based on ALSPAC data, there was little evidence of selection bias in associations of maternal smoking with birthweight, and bias for father/partner smoking was relatively small. Missing partner data can induce selection bias. In our example analyses of the effect of parental smoking on offspring birthweight, the bias had a relatively small impact. In practice, the impact of selection bias will depend on both the analysis model and the selection mechanism.
The human sciences should seek generalisations wherever possible. For ethical and scientific reasons, it is desirable to sample more broadly than ‘Western, educated, industrialised, rich, and democratic’ (WEIRD) societies. However, restricting the target population is sometimes necessary; for example, young children should not be recruited for studies on elderly care. Under which conditions is unrestricted sampling desirable or undesirable? Here, we use causal diagrams to clarify the structural features of measurement error bias and target population restriction bias (or ‘selection restriction’), focusing on threats to valid causal inference that arise in comparative cultural research. We define any study exhibiting such biases, or confounding biases, as weird (wrongly estimated inferences owing to inappropriate restriction and distortion). We explain why statistical tests such as configural, metric and scalar invariance cannot address the structural biases of weird studies. Overall, we examine how the workflows for causal inference provide the necessary preflight checklists for ambitious, effective and safe comparative cultural research.
Chapter 6 introduces the concept of present value and of rate of return analysis as the major tools used by economists to measure returns to investment in human capital. To do this, the discussion introduces the costs of an educational investment and what these consist of, and brings these costs into an analysis of estimating the rate of return to education using two different methods – the “calculated rate” and the “Mincer rate,” including critiques of the Mincer rate. The chapter introduces the concept of social costs and social return, the “option value” of schooling, and further analyzes the problem of “selection bias” – how economists try to “identify” the present value or rate of return to the additional skills learned in school, distinguishing the returns to investing in these skills from other factors that influence the higher wages/earnings of those with more schooling. To illustrate this identification problem, a case study is presented of estimating the returns to education for identical twins with different attainment levels.
As a continuation of the overall introduction to the book, Chapter 2 summarizes the main contributions of economics to understanding the role of education in society and to educational policy. The chapter details these contributions in three parts: (1) economists have demonstrated that education has an important economic dimension (that it has economic value), and they have inserted education policy near the center of the debate on economic development and material well-being; (2) they have formalized concrete models of student learning, both in and out of school, and have developed models of educational production, in which schools, districts, and states are economic decision units, allocating resources to produce educational outputs – and where incentives and resource allocation decisions affect the productivity of teachers and student learning, economists have been able to model a number of strategies that increase output and test them empirically; and (3) economists have also been at the forefront of applying new and increasingly sophisticated statistical techniques to estimate quantitatively the causal effects of various educational policies on student academic outcomes and adult economic and social outcomes.
Chapter 12 focuses on how economists model production functions for education production units and, using these models, estimate the effect various inputs have on student outcomes. The most common educational production models are single output (usually student academic performance as measured by test scores), multi-input, and use secondary data collected at the school/classroom/individual student levels to estimate model parameters. Since these are not experimental data, students are not randomly assigned to inputs, and the main methodological problem is to identify the causal impact of particular inputs on student outcomes. The chapter discusses the role of teachers in educational production functions, the methods economists have used to estimate the contribution of teachers to knowledge production, as well as some examples of models to estimate the causal effects of other inputs into the production process – specifically, computer-assisted learning in primary school, summer school and student retention in primary and middle school, and an increased time on core subject teaching through a longer school day.
Inductive reasoning involves generalizing from samples of evidence to novel cases. Previous work in this field has focused on how sample contents guide the inductive process. This chapter reviews a more recent and complementary line of research that emphasizes the role of the sampling process in induction. In line with a Bayesian model of induction, beliefs about how a sample was generated are shown to have a profound effect on the inferences that people draw. This is first illustrated in research on beliefs about sampling intentions: was the sample generated to illustrate a concept or was it generated randomly? A related body of work examines the effects of sampling frames: beliefs about selection mechanisms that cause some instances to appear in a sample and others to be excluded. The chapter describes key empirical findings from these research programs and highlights emerging issues such as the effect of timing of information about sample generation (i.e., whether it comes before or after the observed sample) and individual differences in inductive reasoning. The concluding section examines how this work can be extended to more complex reasoning problems where observed data are subject to selection biases.
Hospital-based biobanks are being increasingly considered as a resource for translating polygenic risk scores (PRS) into clinical practice. However, since these biobanks originate from patient populations, there is a possibility of bias in polygenic risk estimation due to overrepresentation of patients with higher frequency of healthcare interactions.
Methods
PRS for schizophrenia, bipolar disorder, and depression were calculated using summary statistics from the largest available genomic studies for a sample of 24 153 European ancestry participants in the Mass General Brigham (MGB) Biobank. To correct for selection bias, we fitted logistic regression models with inverse probability (IP) weights, which were estimated using 1839 sociodemographic, clinical, and healthcare utilization features extracted from electronic health records of 1 546 440 non-Hispanic White patients eligible to participate in the Biobank study at their first visit to the MGB-affiliated hospitals.
Results
Case prevalence of bipolar disorder among participants in the top decile of bipolar disorder PRS was 10.0% (95% CI 8.8–11.2%) in the unweighted analysis but only 6.2% (5.0–7.5%) when selection bias was accounted for using IP weights. Similarly, case prevalence of depression among those in the top decile of depression PRS was reduced from 33.5% (31.7–35.4%) to 28.9% (25.8–31.9%) after IP weighting.
Conclusions
Non-random selection of participants into volunteer biobanks may induce clinically relevant selection bias that could impact implementation of PRS in research and clinical settings. As efforts to integrate PRS in medical practice expand, recognition and mitigation of these biases should be considered and may need to be optimized in a context-specific manner.
Small businesses employ more than half of the entire workforce, account for more than sixty percent of new jobs created in the United States, and are responsible for about fifty percent of private domestic gross product. It is noteworthy, however, that small business owners in credit markets, in particular minority owners, have difficulty in securing sources of capital for their business operation. The literature on credit market discrimination shows consistent results that can be interpreted as evidence that minority owners are discriminated against compared to their counterparts (i.e., White owners) in obtaining loans, which may be caused by lenders’ discrimination, although such behavior is prohibited under current fair-lending laws. This paper uses pooled cross-sectional data from the Survey of Small Business Finances (1993, 1998, and 2003) and a bivariate probit model based on James J. Heckman’s approach to deal with sample selection bias for those choosing to apply for loans. Those who didn’t apply for loans have been ignored in analyses of credit markets for small business owners. This paper adds to the small business lending market literature by 1) combining cross sectional data from the Survey of Small Business Finances (SSBF) for 1993, 1998, and 2003 to get more precise estimates and test statistics with more power; 2) conducting regression analyses with different model specifications to show the robustness of the empirical results; and 3) dealing directly with problems of sample selection based on Heckman’s approach with particular attention to the assumptions required to justify the identification of the effect (i.e., exclusion restrictions).
The analysis confirms previous results, suggesting that minority owners are discriminated against in credit markets. These conclusions are supported in a variety of model specifications.
Many popular books and articles that purport to explain how people, companies, orideas succeed highlight a few successes chosen to fit a particular narrative. Weinvestigate what effect these highly selected “success narratives”have on readers’ beliefs and decisions. We conducted a large, randomized,pre-registered experiment, showing participants successful firms with foundersthat all either dropped out of or graduated college, and asked them to makeincentive-compatible bets on a new firm. Despite acknowledging biases in theexamples, participants’ decisions were very strongly influenced by them.People shown dropout founders were 55 percentage points more likely to bet on adropout-founded company than people who were shown graduate founders. Mostreported medium to high confidence in their bets, and many wrote causalexplanations justifying their decision. In light of recent concerns about falseinformation, our findings demonstrate how true but biased information canstrongly alter beliefs and decisions.
Political elites increasingly express interest in evidence-based policymaking, but transparent research collaborations necessary to generate relevant evidence pose political risks, including the discovery of sub-par performance and misconduct. If aversion to collaboration is non-random, collaborations may produce evidence that fails to generalize. We assess selection into research collaborations in the critical policy arena of policing by sending requests to discuss research partnerships to roughly 3,000 law enforcement agencies in 48 states. A host of agency and jurisdiction attributes fail to predict affirmative responses to generic requests, alleviating concerns over generalizability. However, across two experiments, mentions of agency performance in our correspondence depressed affirmative responses – even among top-performing agencies – by roughly eight percentage points. Many agencies that initially indicate interest in transparent, evidence-based policymaking recoil once performance evaluations are made salient. We discuss several possible mechanisms for these dynamics, which can inhibit valuable policy experimentation in many communities.
We emphasize that the ability for a corpus to provide accurate estimates of a linguistic parameter depends on the combined influence of domain considerations (coverage bias and selection bias) and distribution considerations (corpus size). By using a series of experimental corpora on the domain of Wikipedia articles, we can demonstrate the impact of corpus size, coverage bias, selection bias, and stratification on representativeness. Empirical results show that robust sampling methods and large sample sizes can only give you a better representation of the operational domain (i.e. overcome selection bias). However, by themselves, these factors cannot help you achieve accurate quantitative-linguistic analyses for the actual domain (i.e. overcome coverage bias) Uncontrolled domain considerations can lead to unpredictable results with respect to accuracy.
Residential energy efficiency programs play an important role in combating climate change. More precise quantification of the magnitude and timing of energy savings would bring large system benefits, allowing closer integration of energy efficiency into resource adequacy planning and balancing variable renewable electricity. However, it is often difficult to quantify the efficacy of an energy efficiency intervention, because doing so requires consideration of a hypothetical counterfactual case in which there was no intervention, and randomized control trials are often implausible. Although quasi-experimental econometric evaluation sometimes works well, we find that for a set of energy efficiency rebate programs in Northern California, a naïve interpretation of econometric measurement finds that rebate participation is associated with an average increase in electricity consumption of 7.2% [4.5%, 10.1%], varying in magnitude and sign depending on the type of appliance or service covered by the rebate. A subsequent household survey on appliance purchasing behavior and analysis of utility customer outreach data suggest that this regression approach is likely measuring the gross impact of buying a new appliance but fails to adequately capture a counterfactual comparison. Indeed, it is unclear whether it is even possible to construct a suitable counterfactual for econometric analyses of these rebate programs using data generally available to electric utilities. We view these results as an illustration of a limitation of econometric methods of program evaluation and the importance of weighing engineering modeling and other imperfect methods against one another when attempting to provide useful evaluations of real-world policy interventions.
In this chapter, I address the issue of selection bias more directly. First, I present a comparative case study using most-similar research design between the two similar princely states of Awadh and Hyderabad, which shows that historical contingency determined by external geo-political circumstances prevented British from being selective and led to direct rule in Awadh vs. indirect rule in Hyderabad. Second, I develop a new instrument for the British choice of indirect rule through princely states based on the exogenous effect of major European Great Power wars that decreased the ability of the British to fight wars of annexation to bring additional territory into direct rule and increased their tendency to sign treaties of indirect rule with Indian states on the frontiers of British direct rule. The instrumental variable (IV-2SLS) analysis is a major empirical contribution and allows an estimate of the causal effects of colonial indirect rule on Maoist insurgency. I also develop a fine-grained typology of different types of princely states and show that warrior states like Mysore had higher development, while successor states like Hyderabad and feudatory states like Bastar had more inequality and less development and thus Maoist insurgency.
In this chapter, I test the theory of the effect of colonial indirect rule on postcolonial insurgency using an all-India district-level dataset. I use unique Ministry of Home Affairs (MHA) data as a measure of the dependent variable of Maoist control. Unlike other quantitative studies of Maoist insurgency that use measures of violence from 2005 to 2010 as their dependent variable, I do not use such violence data as my dependent variable since violence is only the most visible aspect of insurgency and does not measure actual Maoist rebel control, which is a more multidimensional concept. There is potential bias in the OLS regression estimates since the British possibly selected those districts for indirect rule that were worse off in terms of revenue and productivity. To correctly estimate the causal effect of colonial indirect rule, I control for some observable pre-colonial determinants of indirect rule choice, like forest cover, terrain, soil quality, and pre-colonial agrarian rebellion and find that princely state is still a statistically significant predictor of Maoist control. While these pre-colonial qualities may have played a role in the choice of institutions, once in place such colonial indirect rule had an independent causal effect on postcolonial insurgency.
Detrital zircon geochronology can help address stratigraphic- to lithospheric-scale geological questions. The approach is reliant on statistically robust, representative age distributions that fingerprint source areas. However, there is a range of biases that may influence any detrital age signature. Despite being a fundamental and controllable source of bias, handpicking of zircon grains has received surprisingly little attention. Here, we show statistically significant differences in age distributions between bulk-mounted and handpicked fractions from an unconsolidated heavy mineral sand deposit. Although there is no significant size difference between bulk-mounted and handpicked grains, there are significant differences in their aspect ratio, circularity and colour, which indicate inadvertent preferential visual selection of euhedral and coloured zircon grains. Grain colour comparisons between dated and bulk zircon fractions help quantify bias. Bulk-mounting is the preferred method to avoid human-induced selection bias in detrital zircon geochronology.
A primary challenge for researchers that make use of observational data is selection bias (i.e. the units of analysis exhibit systematic differences and dis-homogeneities due to non-random selection into treatment). This article encourages researchers in acknowledging this problem and discusses how and – more importantly – under which assumptions they may resort to statistical matching techniques to reduce the imbalance in the empirical distribution of pre-treatment observable variables between the treatment and control groups. With the aim of providing a practical guidance, the article engages with the evaluation of the effectiveness of peacekeeping missions in the case of the Bosnian civil war, a research topic in which selection bias is a structural feature of the observational data researchers have to use, and shows how to apply the Coarsened Exact Matching (CEM), the most widely used matching algorithm in the fields of Political Science and International Relations.
In clinical research, populations are often selected on the sum-score of diagnostic criteria such as symptoms. Estimating statistical models where a subset of the data is selected based on a function of the analyzed variables introduces Berkson's bias, which presents a potential threat to the validity of findings in the clinical literature. The aim of the present paper is to investigate the effect of Berkson's bias on the performance of the two most commonly used psychological network models: the Gaussian Graphical Model (GGM) for continuous and ordinal data, and the Ising Model for binary data.
Methods
In two simulation studies, we test how well the two models recover a true network structure when estimation is based on a subset of the data typically seen in clinical studies. The network is based on a dataset of 2807 patients diagnosed with major depression, and nodes in the network are items from the Hamilton Rating Scale for Depression (HRSD). The simulation studies test different scenarios by varying (1) sample size and (2) the cut-off value of the sum-score which governs the selection of participants.
Results
The results of both studies indicate that higher cut-off values are associated with worse recovery of the network structure. As expected from the Berkson's bias literature, selection reduced recovery rates by inducing negative connections between the items.
Conclusion
Our findings provide evidence that Berkson's bias is a considerable and underappreciated problem in the clinical network literature. Furthermore, we discuss potential solutions to circumvent Berkson's bias and their pitfalls.
China’s small-scale agricultural producers face many challenges to increasing productivity and efficiency. In recent years, the Chinese government has provided support for farmer professional cooperatives (FPCs) to connect small farms with upstream and downstream processes in the food supply chain. This study combines propensity score matching and sample selection-corrected stochastic production frontier analysis to estimate the impacts of FPC participation by greenhouse vegetable producers on technical efficiency and income. Results indicate that FPCs help participants improve returns to scale and marginal returns to land and labor, increase technical efficiency, and obtain ¥4,460 (18%) greater income per greenhouse than nonparticipants.