Replication is an important tool used to test and develop scientific theories. Areas of biomedical and psychological research have experienced a replication crisis, in which many published findings failed to replicate. Following this, many other scientific disciplines have been interested in the robustness of their own findings. This chapter examines replication in primate cognitive studies. First, it discusses the frequency and success of replication studies in primate cognition and explores the challenges researchers face when designing and interpreting replication studies across the wide range of research designs used across the field. Next, it discusses the type of research that can probe the robustness of published findings, especially when replication studies are difficult to perform. The chapter concludes with a discussion of different roles that replication can have in primate cognition research.

Our exploration of the 4D Framework uses an eclectic set of methodological techniques; Chapter 3 is an overview of the methodological core of our inquiry. We explain the key operationalizations of the 4D Framework and provide context and details for the studies that appear in multiple chapters throughout the book. We specifically describe how we measure contextual features of a discussion, such as disagreement, as well as the scales we use to measure individual dispositions, such as social anxiety. We explain the utility of psychophysiological data for our purposes and describe the research design details for the studies that use psychophysiological data. We provide details on our survey samples from which several analyses throughout the book are derived.

Chapter 5 describes the process of constructing the MI 1984–2014 Corpus, from compiling the sampling frame (e.g. search terms, dates covered) to the compilation procedure for the illness and year subcorpora. In particular, a detailed discussion of the interpretative status of search terms is provided. Practical issues related to compiling corpora such as cleaning the data are discussed. Furthermore, the problems that the interdisciplinary nature of corpus construction poses for the researcher are outlined.

Two methods are applied to detect differences between corpus (sub )registers, exemplified by the press editorials sections in the British, Canadian and Jamaican components of the International Corpus of English. By design, these methods are apt to target differences between varieties that are represented by putatively comparable corpus material, but it turns out that many of the observed differences can in fact be laid at the door of different sampling strategies applied by corpus compilers. In the example at hand, contrasts can be traced back to the division into institutional and personal editorials. This finding gives rise to a call for a higher granularity of sampling schemes, richer metadata (e.g. on the situational characteristics of the language samples included), and better documentation. As for the methods chosen, the author demonstrates that corpus-driven profiling based either on POS monograms or on higher-level multi-dimensional analysis performs reasonably well, with smaller differences in robustness and computational expense.

Chapter 2 starts by placing experiments in the scientific process – experiments are only useful in the context of well-motivated questions, thoughtful theories, and falsifiable hypotheses. The author then turns to sampling and measurement since careful attention to these topics, despite being often neglected by experimentalists, are imperative. The remainder of Chapter 2 offers a detailed discussion of causal inference that is used to motivate an inclusive definition of “experiments.” The author views this as more than a pedantic exercise, as careful consideration of approaches to causal inference reveals the often implicit assumptions that underlie all experiments. The chapter concludes by touching on the different goals experiments may have and the basics of analysis. The chapter serves as a reminder of the underlying logic of experimentation and the type of mindset one should have when designing experiments. A central point concerns the importance of counterfactual thinking, which pushes experimentalists to think carefully about the precise comparisons needed to test a causal claim.

We show that attending to domain considerations in corpus design involves three steps: (1) describing the domain as fully as possible; (2) operationalizing the domain; (3) sampling the texts. Describing the domain requires defining the boundaries of the domain: what texts belong within the domain and what do not? Describing the domain requires identifying important internal categories of texts that reflect qualitative variation within the domain. Domain description should be carried out systematically using a range of sources that can be evaluated for quality and triangulated. Operationalizing the domain refers to specifying the set of texts that are available for sampling; operational domains are always precisely bounded and specified. A sampling frame is an itemized list of all texts (from the operational domain) that are available for sampling. A sampling unit is the individual “object” (usually a text) that will be included in the corpus. Stratification is the process of collecting texts according to identified categories within the domain, and is usually desirable in corpus design. Proportionality refers to the relative sizes of strata within the sample. Strata can be proportional or equal-sized. Sampling methods can be broadly categorized as random and nonrandom.

The chapter describes the assumptions and goals of large n statistical studies and evaluates the extent to which they generate knowledge about international relations.

Since the objective of our research is to examine the influence of the creative potential of language speakers on their creative performance in the formation and interpretation of new/potential complex words, there are several fundamental methodological principles that have to be taken into consideration. First of all is the method of measuring the creative potential of language speakers and the methods of testing their creative performance Therefore, this chapter introduces the Torrance Test of Creative Thinking, accounts for the basic characteristics of creativity indicators and subscores, and justifies its relevance to our research. Furthermore, it presents a word-formation test and a word-interpretation test and accounts for their objectives and principles of evaluation. A sample of respondents, comprising a group of secondary school students and a group of university students, is introduced. A method of evaluating the data is explained, based on the comparison of two cohorts with opposite scores. Finally, this chapter presents the hypotheses underlying our research.

Chapter 14 develops methods for reliability-based design optimization (RBDO). Three classes of RBDO problems are considered: minimizing the cost of design subject to reliability constraints, maximizing the reliability subject to a cost constraint, and minimizing the cost of design plus the expected cost of failure subject to reliability and other constraints. The solution of these problems requires the coupling of reliability methods with optimization algorithms. Among many solution methods available in the literature, the main focus in this chapter is on a decoupling approach using FORM, which under certain conditions has proven convergence properties. The approach requires the solution of a sequence of decoupled reliability and optimization problems that are shown to gradually approach a near-optimal solution. Both structural component and system problems are considered. An alternative approach employs sampling to compute the failure probability with the number of samples increasing as the optimal solution point is approached. Also described are approaches that make use of surrogate models constructed in the augmented space of random variables and design parameters. Finally, the concept of buffered failure probability is introduced as a measure closely related to the failure probability, which provides a convenient alternative in solving the optimization subproblem.

This chapter presents basic information for a wide readership on how accents differ and how those differences are analyzed, then lays out the sample of performances to be studied, the phonemes and word classes to be analyzed, and the methods of phonetic, quantitative, and statistical analysis to be followed.

This chapter examines skills developed by, and brought to play in, fieldwork. Progressing from generic skills used and refined through fieldwork, the discussion focuses on the geographical nature of skills used across all fieldwork activities, to the key geographical skills and tools that can be drawn upon to construct authentic fieldwork experiences for students. Fieldwork has always been an important facet of geography, helping to inform, validate, and consolidate the study of people and place. Fieldwork remains, to this day, rather simple and straightforward. It involves the gathering of primary data in the field. The ‘process’ of fieldwork occurs through the use and application of a wide variety of geographic and generic skills. The following discussion of fieldwork skills will examine the place of: • fieldwork skills in students wider learning, • fieldwork skills for thinking geographically, • specific geographic fieldwork skills, • geographic fieldwork tools and technology.

Chapter 4 focuses on lay people tackling small-scale reasoning problems in the lab. I identify five main principles that summarize a broad range of empirical findings. I argue that these principles – especially the use of causal thinking and simulation – are basic to human thought and not an optional add-on or rule of thumb. I also discuss the difficulties people have in evaluating their own reasoning processes.

Chapter 1 provides introductory information on empiricism, the stages and elements of the research process; quality criteria; basic types of research questions and methodological approaches, data collection, documentation and analysis, interpretation, reflection and presentation of the findings; and research ethics. Chapter 1 also contains detailed instructions for keeping research diaries and making poster presentations as forms of documentation and presentation which we recommend for student projects, as well as short exercises and recommendations for further reading.