Anchoring has been shown to influence numeric judgments in various domains, including preferential judgment tasks. Whereas many studies and a recent Many Labs project have shown robust effects in classic anchoring tasks, studies of anchoring effects on preferential judgments have had inconsistent results. In this paper, we investigate the replicability and robustness of anchoring on willingness-to-pay, which is a widely used measure for consumer preference. We employ a combination of approaches, aggregating data from previous studies and also contributing additional replication studies designed to reconcile inconsistent previous results. We examine the effect of differing experimental procedures used in prior studies, and test whether publication bias could contribute to the inconsistent findings. We find that different experimental procedures used in previous studies do not explain the divergent results, and that anchoring effects are generally robust to differences in procedures, participant populations, and experimental settings.

People often choose the option that is better on the most subjectively prominent attribute — the prominence effect. We studied the effect of prominence in health care priority setting and hypothesized that values related to health would trump values related to costs in treatment choices, even when individuals themselves evaluated different treatment options as equally good. We conducted pre-registered experiments with a diverse Swedish sample and a sample of international experts on priority setting in health care (n = 1348). Participants, acting in the role of policy makers, revealed their valuation for different medical treatments in hypothetical scenarios. Participants were systematically inconsistent between preferences expressed through evaluation in a matching task and preferences expressed through choice. In line with our hypothesis, a large proportion of participants (General population: 92%, Experts 84% of all choices) chose treatment options that were better on the health dimension (lower health risk) despite having previously expressed indifference between those options and others that were better on the cost dimension. Thus, we find strong evidence of a prominence effect in health-care priority setting. Our findings provide a psychological explanation for why opportunity costs (i.e., the value of choices not exercised) are neglected in health care priority setting.

Prospect Theory (PT: Kahneman & Tversky, 1979) of risky decision making is based on psychological phenomena (paradoxes) that motivate assumptions about how people react to gains and losses, and how they weight outcomes with probabilities. Recent studies suggest that people’s numeracy affect their decision making. We therefore conducted a large-scale conceptual replication of the seminal study by Kahneman and Tversky (1979), where we targeted participants with larger variability in numeracy. Because people low in numeracy may be more dependent on anchors in the form of other judgments we also manipulated design type (within-subject design, vs. single-stimuli design, where participants assess only one problem). The results from about 1,800 participants showed that design type had no effect on the modal choices. The rate of replication of the paradoxes in Kahneman and Tversky was poor and positively related to the participants’ numeracy. The Probabilistic Insurance Effect was observed at all levels of numeracy. The Reflection Effects were not fully replicated at any numeracy level. The Certainty and Isolation Effects explained by nonlinear probability weighting were replicated only at high numeracy. No participant exhibited all 9 paradoxes and more than 50% of the participants exhibited at most three of the 9 paradoxes. The choices by the participants with low numeracy were consistent with a shift towards a cautionary non-compensatory strategy of minimizing the risk of receiving the worst possible outcome. We discuss the implications for the psychological assumptions of PT.

In “The value of nothing: asymmetric attention to opportunity costs drives intertemporal decision making” Read, Olivola and Hardisty (2017) proposed an asymmetric subjective opportunity cost (ASOC) effect to explain and predict why impatience can be detected in intertemporal choice. This work deserves to be replicated and extended for its novel and potentially important findings. The present study aimed to examine the reliability and robustness of the evidence presented by Read et al. by conducting precise replications of their key findings in Study 1. The ASOC effect (Read, et al., 2017) was important for expanding its application and reported to be typically stronger when baseline larger-but-later option (LL) and smaller-but-sooner option (SS) preferences were closer to 50% in the authors’ original condition. Therefore, the present study also aimed to replicate and test the ASOC effect when baseline LL preferences were higher or lower than those in the original condition. We intended to set two additional conditions wherein either LL or SS is more obviously favored (i.e., baseline LL preferences were higher or lower than those in the original condition) by respectively applying the common difference effect (Kirby & Herrnstein, 1995) and the unit effect (Burson, Larrick & Lynch Jr., 2009; Pandelaere, Briers & Lembregts, 2011). Having successfully generated two more obviously favored conditions, the ASOC effect was replicated and confirmed under the original condition and one additional condition wherein SS was more obviously favored. However, the ASOC effect was not detected under the other additional condition wherein LL was more obviously favored. The implications of these findings were discussed.

Bias Blind Spot (BBS) is the phenomenon that people tend to perceive themselves as less susceptible to biases than others. In three pre-registered experiments (overall N = 969), we replicated two experiments of the first demonstration of the phenomenon by Pronin et al. (2002). We found support of the BBS hypotheses, with effects in line with findings in the original study: Participants rated themselves as less susceptible to biases than others (d = –1.00 [–1.33, –0.67]). Deviating from the original, we found an unexpected effect that participants rated themselves as having fewer shortcomings (d = –0.34 [–0.46, –0.23]), though there was support for the target’s main premise that BBS was stronger for biases than for shortcomings (d = –0.43 [–0.56, –0.29]). Extending the replications, we found that beliefs in own free will were positively associated with BBS (r ∼ 0.17–0.22) and that beliefs in both own and general free will were positively associated with self-other asymmetry related to personal shortcomings (r ∼ 0.16–0.24). Materials, datasets, and code are available on https://osf.io/3df5s/.

In 2012, two independent groups simultaneously demonstrated that intuitive mindset enhances belief in God. However, there is now some mixed evidence on both the effectiveness of manipulations used in these studies and the effect of mindset manipulation on belief in God. Thus, this proposal attempted to replicate one of those experiments (Shenhav, Rand & Greene, 2012) for the first time in a high-powered experiment using an under-represented population (Turkey). In line with the intuitive belief hypothesis, a negative correlation between reflectiveness and religious belief emerged, at least in one of the experimental conditions. In contrast to that hypothesis, however, the results revealed no effect of the cognitive style manipulation on religious belief. Although a self-report measure (Faith in Intuition) provided evidence that the manipulation worked as intended, it did not influence actual performance (Cognitive Reflection Test), suggesting a demand effect problem. Overall, the results failed to provide support for the intuitive belief hypothesis in our non-WEIRD sample, despite generally following the predicted patterns, and suggest that using stronger manipulation techniques are warranted in future studies.

Redundant or excessive information can sometimes lead people to lean on it unnecessarily. Certain experimental designs can sometimes bias results in the researcher’s favor. And, sometimes, interesting effects are too small to be studied, practically, or are simply zero. We believe a confluence of these factors led to a recent paper (Isaac & Brough, 2014, JCR). This initial paper proposed a new means by which probability judgments can be led astray: the category size bias, by which an individual event coming from a large category is judged more likely to occur than an event coming from a small one. Our work shows that this effect may be due to instructional and mechanical confounds, rather than interesting psychology. We present eleven studies with over ten times the sample size of the original in support of our conclusion: We replicate three of the five original studies and reduce or eliminate the effect by resolving these methodological issues, even significantly reversing the bias in one case (Study 6). Studies 7–8c suggest the remaining two studies are false positives. We conclude with a discussion of the subtleties of instruction wording, the difficulties of correcting the record, and the importance of replication and open science.

A pioneering study by Loewen et al. made use of the Canadian legislature's newly instituted lottery, which enabled non-cabinet Members of Parliament (MPs) to propose a bill or motion. Their study used this lottery in order to identify the causal effect of proposal power on incumbents' vote share in the next election. Analyzing the first two parliaments to use the lottery, Loewen et al. found that proposal power benefits incumbents, but only incumbents who belong to the governing party. Our study builds on these initial results by adding data from four subsequent parliaments. The pooled results no longer support the hypothesis that MPs—even those who belong to the governing party—benefit appreciably from proposal power. These updated findings resolve a theoretical puzzle noted by Loewen et al., as proposal power would not ordinarily be expected to confer electoral benefits in strong party systems, such as Canada's.

Determination of sample size (the number of replications) is a key step in the design of an observational study or randomized experiment. Statistical procedures for this purpose are readily available. Their treatment in textbooks is often somewhat marginal, however, and frequently the focus is on just one particular method of inference (significance test, confidence interval). Here, we provide a unified review of approaches and explain their close interrelationships, emphasizing that all approaches rely on the standard error of the quantity of interest, most often a pairwise difference of two means. The focus is on methods that are easy to compute, even without a computer. Our main recommendation based on standard errors is summarized as what we call the 1-2-3 rule for a difference of two treatment means.

Ahsan, Sinha, and Srinivasan (2020) studied the motives of knowledge-intensive Indian firms’ international expansion based on resource-based considerations and the locational advantages offered by host countries. They identified firm characteristics associated with strategic asset-seeking, opportunity-seeking, and market-seeking motives. In this replication study, we examine Ahsan et al.'s (2020) model in the Chinese context. Based on our improved empirical model, our findings reveal some similarities but more importantly some key differences in the antecedents of internationalization motives between Indian and Chinese firms. Drawing on insights from prior studies, we propose that these differences can be attributed to differences in absorptive capacity, international expansion scales and patterns, ownership type, and the home institutional contexts in which Indian and Chinese firms operate. Overall, this replication study demonstrates the importance of contextualizing international business research.

We replicate a design ideation experiment (Goucher-Lambert et al., 2019) with and without inspirational stimuli and extend data collection sources to eye-tracking and a think aloud protocol to provide new insights into generated ideas. Preliminary results corroborate original findings: inspirational stimuli have an effect on idea output and questionnaire ratings. Near and far inspirational stimuli increased participants’ idea fluency over time and were rated more useful than control. We further enable experiment reproducibility and provide publicly available data.

In a seminal article published in 2003, Blais et al. demonstrated that local candidates mattered for about 5 per cent of voters in the 2000 Canadian federal election. This study's reliance on a single election raises external validity concerns. We replicate Blais et al.'s original analyses on four elections from 2000 to 2008 using a decade's worth of data from the Canadian Election Study. The local candidate effect first uncovered by Blais et al. is not specific to a single election. Local candidates are a decisive consideration for about 5 to 8 per cent of voters outside Quebec and for about 2 to 5 per cent of voters in Quebec.