Although Mechanical Turk has recently become popular among social scientists as a source of experimental data, doubts may linger about the quality of data provided by subjects recruited from online labor markets. We address these potential concerns by presenting new demographic data about the Mechanical Turk subject population, reviewing the strengths of Mechanical Turk relative to other online and offline methods of recruiting subjects, and comparing the magnitude of effects obtained using Mechanical Turk and traditional subject pools. We further discuss some additional benefits such as the possibility of longitudinal, cross cultural and prescreening designs, and offer some advice on how to best manage a common subject pool.

This paper proposes a revised version of the original Domain-Specific Risk-Taking (DOSPERT) scale developed by Weber, Blais, and Betz (2002) that is shorter and applicable to a {broader range of ages, cultures, and educational levels}. It also provides a French translation of the revised scale. Using multilevel modeling, we investigated the risk-return relationship between apparent risk taking and risk perception in 5 risk domains. The results replicate previously noted differences in reported degree of risk taking and risk perception at the mean level of analysis. The multilevel modeling shows, more interestingly, that within-participants variation in risk taking across the 5 content domains of the scale was about 7 times as large as between-participants variation. We discuss the implications of our findings in terms of the person-situation debate related to risk attitude as a stable trait.

Decision scientists have identified various plausible sources of ideological polarization over climate change, gun violence, national security, and like issues that turn on empirical evidence. This paper describes a study of three of them: the predominance of heuristic-driven information processing by members of the public; ideologically motivated reasoning; and the cognitive-style correlates of political conservativism. The study generated both observational and experimental data inconsistent with the hypothesis that political conservatism is distinctively associated with either unreflective thinking or motivated reasoning. Conservatives did no better or worse than liberals on the Cognitive Reflection Test (Frederick, 2005), an objective measure of information-processing dispositions associated with cognitive biases. In addition, the study found that ideologically motivated reasoning is not a consequence of over-reliance on heuristic or intuitive forms of reasoning generally. On the contrary, subjects who scored highest in cognitive reflection were the most likely to display ideologically motivated cognition. These findings corroborated an alternative hypothesis, which identifies ideologically motivated cognition as a form of information processing that promotes individuals’ interests in forming and maintaining beliefs that signify their loyalty to important affinity groups. The paper discusses the practical significance of these findings, including the need to develop science communication strategies that shield policy-relevant facts from the influences that turn them into divisive symbols of political identity.

Narrow self-interest is often used as a simplifying assumption when studying people making decisions in social contexts. Nonetheless, people exhibit a wide range of different motivations when choosing unilaterally among interdependent outcomes. Measuring the magnitude of the concern people have for others, sometimes called Social Value Orientation (SVO), has been an interest of many social scientists for decades and several different measurement methods have been developed so far. Here we introduce a new measure of SVO that has several advantages over existent methods. A detailed description of the new measurement method is presented, along with norming data that provide evidence of its solid psychometric properties. We conclude with a brief discussion of the research streams that would benefit from a more sensitive and higher resolution measure of SVO, and extend an invitation to others to use this new measure which is freely available.

Much research in cognitive psychology has focused on the tendency to conserve limited cognitive resources. The CRT is the predominant measure of such miserly information processing, and also predicts a number of frequently studied decision-making traits (such as belief bias and need for cognition). However, many subjects from common subject populations have already been exposed to the questions, which might add considerable noise to data. Moreover, the CRT has been shown to be confounded with numeracy. To increase the pool of available questions and to try to address numeracy confounds, we developed and tested the CRT-2. CRT-2 questions appear to rely less on numeracy than the original CRT but appear to measure closely related constructs in other respects. Crucially, substantially fewer subjects from Amazon’s Mechanical Turk have been previously exposed to CRT-2 questions. Though our primary purpose was investigating the CRT-2, we also found that belief bias questions appear suitable as an additional source of new items. Implications and remaining measurement challenges are discussed.

We introduce the Berlin Numeracy Test, a new psychometrically sound instrument that quickly assesses statistical numeracy and risk literacy. We present 21 studies (n=5336) showing robust psychometric discriminability across 15 countries (e.g., Germany, Pakistan, Japan, USA) and diverse samples (e.g., medical professionals, general populations, Mechanical Turk web panels). Analyses demonstrate desirable patterns of convergent validity (e.g., numeracy, general cognitive abilities), discriminant validity (e.g., personality, motivation), and criterion validity (e.g., numerical and non-numerical questions about risk). The Berlin Numeracy Test was found to be the strongest predictor of comprehension of everyday risks (e.g., evaluating claims about products and treatments; interpreting forecasts), doubling the predictive power of other numeracy instruments and accounting for unique variance beyond other cognitive tests (e.g., cognitive reflection, working memory, intelligence). The Berlin Numeracy Test typically takes about three minutes to complete and is available in multiple languages and formats, including a computer adaptive test that automatically scores and reports data to researchers (http://www.riskliteracy.org). The online forum also provides interactive content for public outreach and education, and offers a recommendation system for test format selection. Discussion centers on construct validity of numeracy for risk literacy, underlying cognitive mechanisms, and applications in adaptive decision support.

Most people are caring and will exert great effort to rescue individual victims whose needy plight comes to their attention. These same good people, however, often become numbly indifferent to the plight of individuals who are “one of many” in a much greater problem. Why does this occur? The answer to this question will help us answer a related question that is the topic of this paper: Why, over the past century, have good people repeatedly ignored mass murder and genocide? Every episode of mass murder is unique and raises unique obstacles to intervention. But the repetitiveness of such atrocities, ignored by powerful people and nations, and by the general public, calls for explanations that may reflect some fundamental deficiency in our humanity — a deficiency that, once identified, might possibly be overcome. One fundamental mechanism that may play a role in many, if not all, episodes of mass-murder neglect involves the capacity to experience affect, the positive and negative feelings that combine with reasoned analysis to guide our judgments, decisions, and actions. I shall draw from psychological research to show how the statistics of mass murder or genocide, no matter how large the numbers, fail to convey the true meaning of such atrocities. The reported numbers of deaths represent dry statistics, “human beings with the tears dried off,” that fail to spark emotion or feeling and thus fail to motivate action. Recognizing that we cannot rely only upon our moral feelings to motivate proper action against genocide, we must look to moral argument and international law. The 1948 Genocide Convention was supposed to meet this need, but it has not been effective. It is time to examine this failure in light of the psychological deficiencies described here and design legal and institutional mechanisms that will enforce proper response to genocide and other forms of mass murder.

Although bullshit is common in everyday life and has attracted attention from philosophers, its reception (critical or ingenuous) has not, to our knowledge, been subject to empirical investigation. Here we focus on pseudo-profound bullshit, which consists of seemingly impressive assertions that are presented as true and meaningful but are actually vacuous. We presented participants with bullshit statements consisting of buzzwords randomly organized into statements with syntactic structure but no discernible meaning (e.g., “Wholeness quiets infinite phenomena”). Across multiple studies, the propensity to judge bullshit statements as profound was associated with a variety of conceptually relevant variables (e.g., intuitive cognitive style, supernatural belief). Parallel associations were less evident among profundity judgments for more conventionally profound (e.g., “A wet person does not fear the rain”) or mundane (e.g., “Newborn babies require constant attention”) statements. These results support the idea that some people are more receptive to this type of bullshit and that detecting it is not merely a matter of indiscriminate skepticism but rather a discernment of deceptive vagueness in otherwise impressive sounding claims. Our results also suggest that a bias toward accepting statements as true may be an important component of pseudo-profound bullshit receptivity.

Across many real-world domains, men engage in more risky behaviors than do women. To examine some of the beliefs and preferences that underlie this difference, 657 participants assessed their likelihood of engaging in various risky activities relating to four different domains (gambling, health, recreation, and social), and reported their perceptions of (1) probability of negative outcomes, (2) severity of potential negative outcomes, and (3) enjoyment expected from the risky activities. Women’s greater perceived likelihood of negative outcomes and lesser expectation of enjoyment partially mediated their lower propensity toward risky choices in gambling, recreation, and health domains. Perceptions of severity of potential outcomes was a partial mediator in the gambling and health domains. The genders did not differ in their propensity towards taking social risks. A fifth domain of activities associated with high potential payoffs and fixed minor costs was also assessed. In contrast to other domains, women reported being more likely to engage in behaviors in this domain. This gender difference was partially mediated by women’s more optimistic judgments of the probability of good outcomes and of outcomes being more intensely positive.

Several decision-making models predict that it should be possible to affect real binary choices by manipulating the relative amount of visual attention that decision-makers pay to the two alternatives. We present the results of three behavioral experiments testing this prediction. Visual attention is controlled by manipulating the amount of time subjects fixate on the two items. The manipulation has a differential impact on appetitive and aversive items. Appetitive items are 6 to 11% more likely to be chosen in the long fixation condition. In contrast, aversive items are 7% less likely to be chosen in the long fixation condition. The effect is present for primary goods, such as foods, and for higher-order durable goods, such as posters.

An important open problem is how values are compared to make simple choices. A natural hypothesis is that the brain carries out the computations associated with the value comparisons in a manner consistent with the Drift Diffusion Model (DDM), since this model has been able to account for a large amount of data in other domains. We investigated the ability of four different versions of the DDM to explain the data in a real binary food choice task under conditions of high and low time pressure. We found that a seven-parameter version of the DDM can account for the choice and reaction time data with high-accuracy, in both the high and low time pressure conditions. The changes associated with the introduction of time pressure could be traced to changes in two key model parameters: the barrier height and the noise in the slope of the drift process.

Individual differences in cognitive abilities and skills can predict normatively superior and logically consistent judgments and decisions. The current experiment investigates the processes that mediate individual differences in risky choices. We assessed working memory span, numeracy, and cognitive impulsivity and conducted a protocol analysis to trace variations in conscious deliberative processes. People higher in cognitive abilities made more choices consistent with expected values; however, expected-value choices rarely resulted from expected-value calculations. Instead, the cognitive ability and choice relationship was mediated by the number of simple considerations made during decision making — e.g., transforming probabilities and considering the relative size of gains. Results imply that, even in simple lotteries, superior risky decisions associated with cognitive abilities and controlled cognition can reflect metacognitive dynamics and elaborative heuristic search processes, rather than normative calculations. Modes of cognitive control (e.g., dual process dynamics) and implications for process models of risky decision-making (e.g., priority heuristic) are discussed.

Using capture-recapture analysis we estimate the effective size of the active Amazon Mechanical Turk (MTurk) population that a typical laboratory can access to be about 7,300 workers. We also estimate that the time taken for half of the workers to leave the MTurk pool and be replaced is about 7 months. Each laboratory has its own population pool which overlaps, often extensively, with the hundreds of other laboratories using MTurk. Our estimate is based on a sample of 114,460 completed sessions from 33,408 unique participants and 689 sessions across seven laboratories in the US, Europe, and Australia from January 2012 to March 2015.

According to prospect theory, people overweight low probability events and underweight high probability events. Several recent papers (notably, Hertwig, Barron, Weber & Erev, 2004) have argued that although this pattern holds for “description-based” decisions, in which people are explicitly provided with probability distributions over potential outcomes, it is actually reversed in “experience-based” decisions, in which people must learn these distributions through sampling. We reanalyze the data of Hertwig et al. (2004) and present a replication to determine the extent to which their phenomenon can be attributed to sampling error (a statistical rather than psychological phenomenon) versus underestimation of rare events (i.e., judgmental bias) versus actual underweighting of judged probabilities. We find that the apparent reversal of prospect theory in decisions from experience can be attributed almost entirely to sampling error, and are consistent with prospect theory and the two-stage model of decision under uncertainty (Fox & Tversky, 1998).

This study provides the first comprehensive analysis of individual perceptions of tail risks. It focuses not only on the probability, as has been studied by Nicholas Barberis and others, but also on anticipation of damage. We examine how those perceptions relate to experts’ estimates and publicly available risk information. Behavioural factors—availability bias, threshold models of choice, worry and trust—are found to have a significant impact on risk perceptions. The probability of tail events is overestimated, which is consistent with probability weighting in prospect theory. Potential damage is underestimated, one reason why individuals do not invest in protective measures.

We present a motivational account of the impact of emotion on decision making, termed the feeling-is-for-doing approach. We first describe the psychology of emotion and argue for a need to be specific when studying emotion's impact on decision making. Next we describe what our approach entails and how it relates emotion, via motivation to behavior. Then we offer two illustrations of our own research that provide support for two important elements in our reasoning. We end with specifying four criteria that we consider to be important when studying how feeling guides our everyday doing.

We conducted an analysis of the 13-item Maximization Scale (Schwartz et al., 2002) with the goal of establishing its factor structure, reliability and validity. We also investigated the psychometric properties of several proposed refined versions of the scale. Four sets of analyses are reported. The first analysis confirms the 3-part factor structure of the scale and assesses its reliability. The second analysis identifies those items that do not perform well on the basis of internal, external, and judgmental criteria, and develops three shorter versions of the scale. In the third analysis, the three refined versions of the scale are cross-validated to confirm dimensionality, reliability, and validity. The fourth analysis uses an experiment in an investment decision making context to assess the reliability and nomological validity of the refined scales. These analyses lead us to conclude that a shorter, 6-item Maximization Scale performs best and should be used by future researchers. It is hoped that clarification of the conceptual underpinnings of the maximization construct and development of a refined scale will enhance its use among researchers across several of the social science disciplines.

We claim that understanding human decisions requires that both automatic and deliberate processes be considered. First, we sketch the qualitative differences between two hypothetical processing systems, an automatic and a deliberate system. Second, we show the potential that connectionism offers for modeling processes of decision making and discuss some empirical evidence. Specifically, we posit that the integration of information and the application of a selection rule are governed by the automatic system. The deliberate system is assumed to be responsible for information search, inferences and the modification of the network that the automatic processes act on. Third, we critically evaluate the multiple-strategy approach to decision making. We introduce the basic assumption of an integrative approach stating that individuals apply an all-purpose rule for decisions but use different strategies for information search. Fourth, we develop a connectionist framework that explains the interaction between automatic and deliberate processes and is able to account for choices both at the option and at the strategy level.

A four-fold classification of emotions with respect to their functions in decision making is proposed. It is argued that emotions are not homogenous concerning their role in decision making, but that four distinct functions can be distinguished concerning emotional phenomena. One function is to provide information about pleasure and pain for preference construction, a second function is to enable rapid choices under time pressure, a third function is to focus attention on relevant aspects of a decision problem, and a fourth function is to generate commitment concerning morally and socially significant decisions. The pertinent literature on the relationship between emotion and decision making is reviewed, and it is concluded that most approaches fit into the proposed framework. We argue that a precise conceptualization of emotional phenomena is required to advance our understanding of the complex role of emotions in decision making.

Research in judgment and decision making generally ignores the distinction between factual and subjective feelings of ownership, tacitly assuming that the two correspond closely. The present research suggests that this assumption might be usefully reexamined. In two experiments on the endowment effect we examine the role of subjective ownership by independently manipulating factual ownership (i.e., what participants were told about ownership) and physical possession of an object. This allowed us to disentangle the effects of these two factors, which are typically confounded. We found a significant effect of possession, but not of factual ownership, on monetary valuation of the object. Moreover, this effect was mediated by participants’ feelings of ownership, which were enhanced by the physical possession of the object. Thus, the endowment effect did not rely on factual ownership per se but was the result of subjective feelings of ownership induced by possession of the object. It is these feelings of ownership that appeared to lead individuals to include the object into their endowment and to shift their reference point accordingly. Potential implications and directions for future research are discussed.