The authors apply logistic regression, multinomial regression, classification trees and random forests to a ternary outcome variable: the variation between the ’s-genitive, the of-genitive and functionally equivalent noun + noun combinations. The statistical approaches discussed fall into regression models on the one hand and classification trees on the other. Specifically, as an alternative to successive binomial regression analyses, the authors implement a multinomial model, which can analyse the entire dataset with three outcome categories simultaneously. Further, a basic classification tree is calculated alongside a more complex (and more robust) random forest. The chapter does not only weigh advantages and shortcomings of all four models, but it also explicates the different rationales and interpretations that come with them. As a major insight, it emerges that the nature of the dataset, the analytic purpose and the statistical model are interdependent and condition each other in several non-trivial respects.

We demonstrate that there is little consensus on what representativeness is, either in statistics or in corpus linguistics. Representative is a general term that must be made specific within a particular context in order to evaluate a sample. We introduce ten attested conceptualizations of corpus representativeness: (1) representativeness as “general acclaim for data”; (2) a representative corpus has been collected with the “absence of selective focus”; (3) a representative corpus contains texts that are “typical or ideal cases” of the target domain; (4) a representative corpus is a “miniature of the population”; (5) a representative corpus achieves “coverage of the population’s heterogeneity”; (6) a representative corpus “permits good estimation”; (7) a representative corpus is a corpus that is “good enough for a particular purpose”; (8) a large corpus is more important than a representative corpus; (9) a representative corpus is a “balanced” corpus; (10) a representative corpus is never possible. The term “balance” does not have a single agreed-upon definition in CL, and in fact, is often defined in contradictory ways. A unified and operational definition of corpus representativeness is needed.

We emphasize that the ability for a corpus to provide accurate estimates of a linguistic parameter depends on the combined influence of domain considerations (coverage bias and selection bias) and distribution considerations (corpus size). By using a series of experimental corpora on the domain of Wikipedia articles, we can demonstrate the impact of corpus size, coverage bias, selection bias, and stratification on representativeness. Empirical results show that robust sampling methods and large sample sizes can only give you a better representation of the operational domain (i.e. overcome selection bias). However, by themselves, these factors cannot help you achieve accurate quantitative-linguistic analyses for the actual domain (i.e. overcome coverage bias) Uncontrolled domain considerations can lead to unpredictable results with respect to accuracy.

We show that attending to domain considerations in corpus design involves three steps: (1) describing the domain as fully as possible; (2) operationalizing the domain; (3) sampling the texts. Describing the domain requires defining the boundaries of the domain: what texts belong within the domain and what do not? Describing the domain requires identifying important internal categories of texts that reflect qualitative variation within the domain. Domain description should be carried out systematically using a range of sources that can be evaluated for quality and triangulated. Operationalizing the domain refers to specifying the set of texts that are available for sampling; operational domains are always precisely bounded and specified. A sampling frame is an itemized list of all texts (from the operational domain) that are available for sampling. A sampling unit is the individual “object” (usually a text) that will be included in the corpus. Stratification is the process of collecting texts according to identified categories within the domain, and is usually desirable in corpus design. Proportionality refers to the relative sizes of strata within the sample. Strata can be proportional or equal-sized. Sampling methods can be broadly categorized as random and nonrandom.

We show that empirical corpus-based research is prevalent across subdisciplines of (applied) linguistics, not just in “corpus linguistics” journals. We define a corpus as a large, principled sample of texts designed to represent a target domain of language use. Corpus representativeness is conceptualized as the extent to which a corpus permits accurate and meaningful generalizations about linguistic patterns that are typical in a domain. Corpus representativeness involves two main considerations, which are both relative to the linguistic research goal of interest: domain considerations (adequate representation of the text varieties in the domain), and distribution considerations (adequate representation of the distribution of linguistic features in the domain).

As discussed in Chapter 1, corpus representativeness depends on two sets of considerations: domain considerations and distribution considerations. Domain considerations focus on describing the arena of language use, and operationally specifying a set of texts that could potentially be included in the corpus. The linguistic research goal, which involves both a linguistic feature and a discourse domain of interest, forms the foundation of corpus representativeness. Representativeness cannot be designed for or evaluated outside of the context of a specific linguistic research goal. Linguistic parameter estimation is the use of corpus-based data to approximate quantitative information about linguistic distributions in the domain. Domain considerations focus on what should be included in a corpus, based on qualitative characteristics of the domain. Distribution considerations focus on how many texts should be included in a corpus, relative to the variation of the linguistic features of interest. Corpus representativeness is not a dichotomy (representative or not representative), but rather is a continuous construct. A corpus may be representative to a certain extent, in particular ways, and for particular purposes.

We propose that the representativeness of a corpus directly depends on its suitability for a specific research goal (including the domain and the linguistic feature(s) of interest). Creating a new corpus involves establishing linguistic research question(s), addressing domain considerations, including describing the domain, operationalizing the domain, evaluating the operational domain (relative to the full domain), designing the corpus, and evaluating the corpus (relative to the operational domain), addressing distribution considerations, including defining a linguistic variable and evaluating the required sample size, collecting the corpus, and documenting and reporting corpus design and representativeness. The steps for evaluating an existing corpus are similar: establishing linguistic research question(s), identifying and acquire the corpus and its documentation, addressing domain considerations, including describing the domain and evaluating the operational domain relative to the full domain, and the corpus relative to the operational domain, addressing distribution considerations, including defining a linguistic variable and evaluating the required sample size, and documenting and reporting corpus design and representativeness. We conclude the book by arguing that corpus representativeness is important for both corpus designers/builders, and corpus researchers who need to evaluate whether a corpus is appropriate for their research goals.

We define a linguistic distribution as the range of values for a quantitative linguistic variable across the texts in a corpus. An accurate parameter estimate means that the measures based on the corpus are close to the actual values of a parameter in the domain. Precision refers to whether or not the corpus is large enough to reliably capture the distribution of a particular linguistic feature. Distribution considerations relate to the question of how many texts are needed. The answer will vary depending on the nature of the linguistic variable of interest. Linguistic variables can be categorized broadly as linguistic tokens (rates of occurrence for a feature) and linguistic types (the number of different items that occur). The distribution considerations for linguistic tokens and linguistic types are fundamentally different. Corpora can be “undersampled” or “oversampled” – neither of which is desirable. Statistical measures can be used to evaluate corpus size relative to research goals – one set of measures enables researchers to determine the required sample size for a new corpus, while another provides a means to determine precision for an existing corpus. The adage “bigger is better” aptly captures our best recommendation for studies of words and other linguistic types.