1 Introduction

At first glance, combining the vast and diverse areas of quantitative and computational phonology in a single survey seems like a daunting task.Footnote ¹ But even a little reflection reveals the necessity of this combination, as establishing an agreed-upon dividing line between these areas is neither possible nor particularly useful. For example, what makes an approach to phonology a “computational” one? Perhaps it means the use of software to implement and/or test a particular model, analysis, or learning algorithm. But this definition omits or at least misconstrues work grounded in a formal theory of the nature of computation that maintains a distinction between algorithm and code. Taking this idea even further, we might consider all work on phonology to be computational in nature, given that the phonology itself is a computational system that solves a variety of problems such as recognition (parsing an overt form), generation (mapping an underlying form to a surface form), and membership (assessing well-formedness). Furthermore, since a phonological grammar is the end target of acquisition, any work that theorizes or characterizes that grammar relates to some version of the phonological learning problem.

The term “quantitative” likewise invokes multiple associations, from the gradient nature of phonological patterns themselves, to the stochastic or nondeterministic algorithms used to model them, to the statistical methods that compare the resulting models to each other. In practice, research grounded in all of these ideas is of course aided by computational tools. The strategy behind this Element, then, is to instead embrace this entanglement and to celebrate the innovation and enthusiasm of scholars who have applied algorithmic, formal, mathematical, statistical, and/or probabilistic methods to the study of phonology and the computational problems it solves. This goal of course provides an enormous amount of ground to cover, and so an additional inclusion criterion is a shared assumption of an abstract phonological grammar in the generative tradition (broadly construed) that is distinct from – though not necessarily independent of – the phonetics. As a result, certain lines of work that are unequivocally computational and/or quantitative are regrettably being left out in the interest of space limitations and narrative cohesion. These include (among others) the learning of phonetic categories (e.g.,Dillon et al. Reference Dillon, Dunbar and Idsardi2013; Thorburn et al. Reference Thorburn, Lau, Feldman, Gong and Kpogo2022; Matusevych et al. Reference Matusevych, Schatz, Kamper, Feldman and Goldwater2023), the modeling of lexical acquisition via phonetic variation (e.g., Elsner et al. Reference Elsner, Goldwater, Eisenstein, Li, Lin, Osborne, Lee and Park2012, Reference Elsner, Goldwater, Feldman, Wood, Yarowsky, Baldwin, Korhonen, Livescu and Bethard2013), and usage-based and exemplar models of phonology (e.g., Bybee Reference Bybee2001, Reference Bybee2007; Pierrehumbert Reference Pierrehumbert, Bybee and Hopper2001a, Reference Pierrehumbert2001b).

For the areas that will be covered, the objective is to be representative rather than exhaustive, in order to demonstrate the myriad ways that quantitative and computational approaches have been applied to a variety of questions grounded in a variety of theoretical perspectives. To that end, the organizational structure is a combination of methods, theories, and problems of interest. Specifically, the outline of the Element is as follows. Section 2 discusses computational work on rule-based phonological grammars and how they are learned. Section 3 then reviews work on constraint-based phonology, with a focus on computational and probabilistic models for the learning of grammars and hidden structure.

Following these discussions of work grounded in phonological grammars of particular types, the next three sections show how these traditional core areas of research have been extended to address additional areas of interest through a variety of methods. Section 4 turns to a central area where quantitative methods have been employed: gradient acceptability and the contribution of lexical statistics to phonotactic generalizations. Section 5 briefly surveys the application of information theoretic methods to questions about phonological structure, and then Section 6 likewise briefly surveys the past, current, and potential future applications of connectionist models to the study of phonology. Stepping back, Section 7 discusses the contributions that formal language and model theoretic approaches to phonology can and have made to the study of phonological typology, learning, and representations. Rather than focusing on particular types of grammars, such work draws distinctions based on phonological patterns themselves, making its findings relevant to any theory of the phonological grammar and how it is learned. Finally, Section 8 concludes.

Importantly, these sectional groupings are not intended as a partition, as the boundary lines (both conceptual and methodological) are often blurry. The broader goal is instead to highlight how these categories complement each other and have great potential for further integration. These problems are hard. Studying them from as many angles as possible can only lead to greater collective progress on the many fascinating open questions about the phonological component of human languages.

2 Rule-Based Phonology

We begin with a discussion of formal approaches to rule-based phonology, a term most often defined in contrast to constraint-based phonology, which will be discussed in Section 3. The distinction between these categories is not absolute, as rule-based theories can and have made use of constraints (e.g., morpheme structure constraints, Halle Reference Halle1959, Stanley Reference Stanley1967, or constraints that block or trigger a rule’s application; see Reiss Reference Reiss, Vaux and Nevins2008 for examples and discussion). A set of rules can also function like constraints on input–output mappings, as in Koskenniemi’s (Reference Koskenniemi1983) theory of two-level rules. Furthermore, the term rule is sometimes used to refer to a context-dependent pattern of alternation between segments. To “learn a phonological rule” can mean to learn the fact that, for example, {t, d} in American English are flapped between two vowels, or {p, t, k} are aspirated as simple onsets. The use of “rule” here is just a shorthand for a type of pattern and is not necessarily tied to any particular assumption about the form of the phonological grammar.

The scope of this section includes work that does make such an assumption, namely that the phonological grammar consists of a set of ordered context-sensitive rules. Two main threads of research will be discussed. The first is work on formalizing such grammars, including specifying the algorithm by which a rule applies to a string. The second is work on the learning of these grammars.

Arguably, the first theory that comes to mind in the context of rule-based phonology is the one proposed in Chomsky and Halle’s (Reference Chomsky and Halle1968) Sound Pattern of English (hereafter SPE), in which phonological rules take the form in (1):

1. (1) $\text{A}\to \text{B}/\text{X}\_\_\_\text{Y}$

This rule asserts that A is rewritten as B when in the context X__Y (i.e., the string XAY is rewritten XBY). However, the extension of such a rule (i.e., the set of input–output string pairs it represents) is ambiguous without a specification for how the rule applies to a string. This ambiguity is most apparent with rules and strings that have the potential for multiple applications. Consider the rule in (2a) and the string in (2b); depending on how the rule applies, the output for this string could be [par], [pa], etcetera.

1. 1. a. $\left[+\text{cons}\right]\to Ø/\_\_\_\#$

   2. b. /park/

The rule-application algorithm specified in SPE has become known as simultaneous application and is stated as follows: “To apply a rule, the entire string is first scanned for segments that satisfy the environmental constraints of the rule. After all such segments have been identified in the string, the changes required by the rule are applied simultaneously” (Chomsky and Halle Reference Chomsky and Halle1968: 344). Assuming this algorithm, the rule in (2a) maps /park/ to [par], since in the underlying representation (UR) only /k/ satisfies the rule’s structural description ([+cons]#).

Johnson (Reference Johnson1972) provides a formal argument for why simultaneous rules are preferable to iterative ones, where an iterative rule is defined as one that applies repeatedly to a string as long as any targets remain. Specifically, he proves that iterative rules can simulate any arbitrary string rewriting system, while simultaneous ones are limited to the power of finite-state transducers (FSTs). He then goes on to develop a theory of linear rules, which – like iterative rules – can apply multiple times to a string, but – like simultaneous rules – are limited to the expressivity of finite-state. Their restrictiveness comes from a requirement that each successive application moves further into the string, in contrast to iterative rules for which there is no imposed order on the different applications. In other words, linear rules are directional and are specified to apply either left to right or right to left.

Despite their formal equivalence, linear rules are argued to be preferable to simultaneous ones on account of the greater simplicity with which they can capture multiple applications. Simultaneous rules do allow for multiple applications through the mechanism of parenthesis-star, by which a rule becomes an abbreviation for an infinite set of rules that contain zero or more tokens of the expression in parentheses. Consider the example ATR harmony rule in (3a). Simultaneous application will only identify the first /ʊ/ as a target in (3b), with the resulting output [putukʊ].

1. 1. a. $\text{V}\to \left[+\text{ATR}\right]/\left[+\text{ATR}\right]{\text{C}}_{\text{0}}\_\_\_$

   2. b. $/\text{put}\mho \text{k}\mho /\to \left[\text{putuk}\mho \right]$

If the actual surface form reflects an additional application (i.e., [putuku]), the rule can be instead formulated as in (4a). This version represents an infinite set of rules, including (3a) as well as (4b) and (4c), that allows for zero or more [−ATR] vowels to intervene between the trigger and target. As long as a vowel satisfies one of the rules in this infinite expansion in the UR, it will harmonize to [+ATR] under simultaneous application (i.e., the second /ʊ/ in (3b) satisfies rule (4b)).

1. 1. a. $\text{V}\to \left[+\text{ATR}\right]/\left[+\text{ATR}\right]{\text{C}}_{\text{0}}\left(\left[-\text{ATR}\right]{\text{C}}_{\text{0}}\right){}^{*}\_\_\_$

   2. b. $\text{V}\to \left[+\text{ATR}\right]\text{/}\left[+\text{ATR}\right]{\text{C}}_{\text{0}}\left[-\text{ATR}\right]{\text{C}}_{\text{0}}\_\_\_$

   3. c. $\text{V}\to \left[+\text{ATR}\right]\text{/}\left[+\text{ATR}\right]{\text{C}}_{\text{0}}\left[-\text{ATR}\right]{\text{C}}_{\text{0}}\left[-\text{ATR}\right]{\text{C}}_{\text{0}}\_\_\_$

As a linear rule that applies left to right, however, (3a) can achieve the same effect without the added notation of parenthesis-star. After the first application generates [putukʊ], the newly created [u] can serve as the trigger for the final vowel. Johnson’s work thus provided a formal grounding from which to compare theories of rules and rule application in terms of both descriptive adequacy and elegance.Footnote ²

We turn now to the question of learning rules and rule-based grammars. From the perspective of formal learnability, such grammars do not have many inherent advantages. Context-sensitive grammars like SPE are not learnable from positive data; indeed (as will be discussed further in Section 7), not even regular grammars are, despite being situated well below context-sensitive on the Chomsky–Schützenberger hierarchy (Chomsky Reference Chomsky1959; Chomsky and Schützenberger Reference Chomsky and Schützenberger1959):

1. (5) Finite $\subset$ Regular $\subset$ Context-free $\subset$ Context-sensitive $\subset$ Recursively enumerable

Early work on rule learning then focused on the question of how a learner can identify the correct rule or grammar when more than one are descriptively adequate. Johnson (Reference Johnson and Wilks1984) demonstrates a deductive approach to learning a limited set of phonological rules of the form in (6), where “a” and “b” are segments and “C” is a feature matrix of unspecified length that is a subset of the segments surrounding “a.”

1. (6) $\text{a}\to \text{b}/\text{C}$

The input data is a set of paradigms with stems inflected with various affixes. Through inspection of this data, the learner identifies the contexts in which “a” alternates with “b” without including those tokens of “a” that do not alternate. In the case of multiple ordered rules, it entertains all possible hypotheses for which alternation took place last, undoes that discovered rule, and then repeats this procedure until all alternations have been accounted for. The procedure is error driven in the sense that it rejects a hypothesis when it arrives at a point where the rule discovery procedure fails. This working-backward technique means the learner will also propose URs for the stems and affixes present in the data.

Johnson’s approach demonstrates that the presence of non-surface-true patterns resulting from rule ordering is not inherently a barrier to learning, but additional selection criteria are needed for such a learner to converge on a single grammar. In a case in which the rules are not strictly ordered (i.e., multiple orderings will correctly generate the data), the learner will identify multiple grammars with no way of deciding among them. Similarly, given that the context of a single rule will not always be strongly data determined, the learner will need to appeal to some type of evaluation metric or guiding principle to select among a set of adequate contexts.

Other work that addresses the learnability of rules has been couched in a substance-free theory that emphasizes the formal nature of phonology as a computational system that operates over symbolic representations. Reiss (Reference Reiss, Vaux and Nevins2008: 258–9) – summarizing views presented in earlier work including Hale and Reiss (Reference Hale and Reiss2000a, Reference Hale, Reiss, Burton‐Roberts, Carr and Docherty2000b) – articulates the substance-free view of the phonological system as follows:

This focus on phonological computation enables a streamlined conception of universal grammar (UG) that facilitates formal discussions and treatments of learning problems. For example, Dell (Reference Dell1981) discusses the subset problem inherent to the learning of optional rules. As an example, French optionally deletes an /l/ that follows an obstruent and precedes a consonant or pause:

1. 1. a. $/\text{kεltabl}/\to \left[\text{kεltabl}\right]~\left[\text{kεltab}\right],quelletable?$ , “which table?”

   2. b. $/\text{parl}/\to \left[\text{parl}\right]\left({}^{*}\left[\text{par}\right]\right)$ , parle, “speak”

Consider two versions of this deletion rule: Deletion A targets coda /l/’s following obstruents, and Deletion B targets coda /l/’s following any consonant. The evidence that distinguishes these two rules is negative evidence, such as the ungrammaticality of *[par] in (7b). But given that such evidence is unavailable to the French-learning child, how do they come to select Deletion A instead of Deletion B? This is the subset problem. The grammar that includes the less restrictive Deletion B will correctly generate all of the observed data. Without negative evidence, the learner will never have reason to consider a more restrictive grammar (i.e., one that generates a proper subset of the forms generated by its current grammar). Dell then proposes that the language acquisition device (LAD) must include the strategy of always selecting the more restrictive grammar when faced with this choice.

Notably, this scenario is only relevant to the case of optional rules. As an optional rule, Deletion A generates the language including [tabl], [tab], and [parl], while the less restrictive Deletion B generates [tabl], [tab], [parl], and *[par]. But if the rules are obligatory, the subset–superset relationship no longer holds: Deletion A generates [tab] and [parl] while Deletion B generates [tab] and *[par]. In this case, positive data – specifically, encountering [parl] – will be sufficient to correct an earlier hypothesis of Deletion B. In this way, obligatory rules serve to provide indirect negative evidence (i.e., if [parl] is grammatical, then *[par] must not be). The challenge, of course, is how the learner can know whether the rule they are learning is optional or obligatory. To address this issue, Dell further proposes that the learner assumes the rule is obligatory until they encounter evidence of optionality.

Hale and Reiss (Reference Hale and Reiss2008) recast the subset principle as a description of the learner’s initial state, rather than a guiding principle for selecting among competing grammars. In particular, they argue that UG must provide the complete set of primitives (i.e., features) so that the learner can posit the most specified rule possible (contra typically assumed pressures of economy in rule formulations) that accounts for all and only the forms to which it applies. Generalizing over observed instances of the rule is achieved via set intersection over the fully specified feature bundles. Maximally specifying while still generalizing is necessary for the reasons stated earlier in this Element: an overly general rule will never be contradicted by positive evidence alone.

This principle of generalizing over more specified instances was employed in the minimal generalization learner of Albright and Hayes (Reference Albright, Hayes and Maxwell2002, Reference Albright and Hayes2003), who investigate rule learning in the context of the rules-versus-analogy debate of inflectional morphology (e.g., Pinker and Prince Reference Pinker, Prince, Lima, Corrigan and Iverson1994; Bybee Reference Bybee2001).Footnote ⁴ The learner compares pairs of (present, past) forms to identify the change that derives the past from the present. For example, the comparison of shine and shined reveals the rule in (8):

1. (8) $Ø\to \text{d}/{\int }^{}\text{a}ɪ\text{n}\_\_\_$

This rule is, of course, overly specific, but additional comparisons reveal more rules that share its structural change (e.g., grab-grabbed, hug-hugged, fill-filled, etc.). The specifications of these rules can be combined into the more general rule in (9a), or using features, (9b).

1. 1. a. $Ø\to \text{d}/\left\{\text{n},\text{b},\text{g},\text{l}\right\}\_\_\_$

   2. b. $Ø\to \text{d}/\left[+\text{cons},+\text{voice}\right]\_\_\_$

This procedure will generate a set of rules that differ in their generality, such that a given input may be subject to more than one rule. To address this ambiguity, all rules are also given a confidence score, which is the number of forms in the training data that the rule correctly applies to (i.e., its number of hits) divided by the number of forms it can apply to (i.e., the rule’s scope). The set of rules along with their confidence scores can account for gradience (see Section 4) if an output’s well-formedness is taken as the score of the “best” rule that generates it.

Getting back to the question of the learner’s initial state, Hale and Reiss’s (Reference Hale and Reiss2008) argument is that because the distinctions needed to work out the phonological system can only be detected to the extent that parsing encodes forms differently, the maximally specified initial representation needed to avoid the subset problem is only possible if UG provides all features. But Odden (Reference Odden2022) rejects this assumption in favor of radical substance-free phonology, in which only the abstract concept of a feature is provided by UG rather than an actual feature set. The learning of both the feature set and the rules of the grammar is guided by the evidence of how and which sounds pattern together (see also Mielke Reference Mielke2008).Footnote ⁵ Even though the phonological system is responsible for parsing those sounds that are recognized as linguistic objects, the auditory system deals with acoustic representations of all sounds. If the current hypothesis for the phonological grammar has discarded important information about, for example, which sounds contrast, that information is still available through the auditory system and so a correction can be made. Given that, the learner’s objective can be characterized by something other than avoiding the subset problem. Odden’s (Reference Odden2022: 526) proposal instead emphasizes simplicity: “The task of feature acquisition is finding the simplest system of properties that accounts for those cases of grammatical functioning-together that can be observed in the primary linguistic data.”

The significance of both restrictiveness and simplicity is recognized by Rasin et al. (Reference Rasin, Shefi, Katzir, Asatryan, Song and Whitmal2020, Reference Rasin, Berger, Lan, Shefi and Katzir2021), who argue that the learner’s task is actually to find the optimal balance between these potentially conflicting demands. Specifically, they propose the use of minimum description length (MDL) for learning not just a grammar of ordered rewrite rules but also the lexicon of URs.Footnote ⁶ Length in this case is the combined total (in bits) of the grammar itself as well as the encoding of the data given that grammar. Consider again the example of optional /l/ deletion in French. The context-free version of this rule in (10a) is shorter than the target version in (10b), and so would be favored by simplicity.

1. 1. a. $\text{l}\to Ø$

   2. b. $\text{l}\to Ø/\left[-\text{son}\right]\_\_\_\#$

Of course (10a) will over-generate, but as discussed earlier in this section, the learner cannot recover from that error without negative evidence. Employing something like the subset principle is necessary to select the correct rule.

However, Rasin et al. argue that the optionality of the rule creates a further problem. If the learner has only encountered one of the possible rule outputs – say either [tab] or [tabl] for “table” – then a grammar that does not generalize to predict the other form will be more restrictive than the one that includes (10b). One such grammar would be the one that does not propose a rule at all, but instead just lists in the lexicon all of the forms observed so far. In this way, the subset principle’s insistence on restrictiveness can have the effect of preventing generalization.

Minimum description length balances these competing demands by computing not just the size of the grammar, but the “cost” of describing the data with that grammar. In particular, the cost of choosing a UR from the lexicon increases with the size of the lexicon (2 choices = 1 bit, 4 choices = 2 bits, etc.), and so generalizing with a rule is cheaper. In addition, whether or not an optional rule has applied to a UR that it can apply to is specified with an additional bit. (This added bit is not needed for obligatory rules, because whether or not they apply is determined by the choice of UR itself.) As an optional rule, then, (10b) is actually cheaper than (10a) because it can apply to fewer URs.

As the recency of Rasin et al.’s (Reference Rasin, Shefi, Katzir, Asatryan, Song and Whitmal2020, Reference Rasin, Berger, Lan, Shefi and Katzir2021) work shows, the question of how to learn rule-based grammars has not been abandoned, but it is undeniable that the progress in phonological learning research was greatly accelerated following the shift to constraint-based grammars like optimality theory (OT) (Prince and Smolensky Reference Prince and Smolensky1993, Reference Prince and Smolensky2004). The next section turns to the research on these grammar types, the comparisons among them, and how to learn them.

3 Constraint-Based Phonology

Constraint-based phonology has become an umbrella term for a collection of phonological theories that centralize constraints on representations instead of the procedures by which those representations are changed.Footnote ⁷ Given how readily these theories lend themselves to computational learning models and statistical methods for working with quantitative data and patterns, it is not surprising that the shift in the field from rule- to constraint-based phonology corresponded to a surge in research on these areas that continues today. This section will begin with an overview of different types of constraint-based grammars (Section 3.1), followed by a survey of the work addressing a variety of phonological learning problems (Section 3.2). Lastly, Section 3.3 will highlight some of the arguments used to compare these theories to each other, including their ability to address questions of theoretical interest as well as what is known about their respective complexity and learnability.

4 Gradient Acceptability and Lexical Statistics

It has long been recognized that speakers have knowledge of which sound sequences are and are not possible in their languages (i.e., phonotactic knowledge). A classic example is blick versus *bnick; neither is an actual word of English, but speakers reliably recognize that only the former is a possible word (Chomsky and Halle Reference Chomsky and Halle1965). Put another way, blick is treated as an accidental gap in the lexicon, whereas *bnick is prohibited from the lexicon due to some type of grammatical constraint. The observed contrast between pairs like blick/*bnick suggests a categorical model in which words are either allowed or disallowed based on their particular sequencing of sounds, and the presence of a single disallowed sequence is enough to condemn the entire word. Indeed blick and bnick are nearly identical, so they cannot be bad across the board. Rather, the problem with *bnick is isolated to a subword component, namely the sequence #bn.Footnote ²⁵

Coleman (Reference Coleman1996) tested the psychological reality of such constraints by collecting acceptability judgments from English speakers for matched nonce words with and without a phonotactic violation (e.g., *mlisless versus glisless). The task was categorical (e.g., forced choice response to whether the word is or could be a word of English), and a word’s overall rating was calculated as the proportion of participants who accepted it. Contrary to the predictions of the categorical model, the results indicated that a single disallowed subword component does not in fact make the word completely unacceptable. In addition, words that did not violate any constraints but were made up of low-frequency components were rated lower than words with high-frequency components, despite both word types being fully grammatical.

This effect of frequency is taken as evidence that speakers are not just aware of what is and is not possible, but can draw on more detailed knowledge of distributional patterns to assess the gradient acceptability of actual and novel words. But this in turn raises the question of which lexical statistics best account for this knowledge (i.e., what frequencies are speakers attuned to). A baseline approach defines phonotactic probability with an n-gram model in which probabilities are assigned to sequences of length n based on their frequency in a training corpus. For example, in a bigram model (n = 2), the probability of the sequence bl is calculated as in (29),

1. (29) $P\left(bl\right)=\frac{C\left(bl\right)}{C\left(b\right)}$

where C(bl) is the number of times bl appears in the corpus and C(b) is the number of times b appears (i.e., the unigram count of b).Footnote ²⁶ The probability of an entire word is the product of the probabilities of its component bigrams. The difference in acceptability between blick and *bnick then would be accounted for if $P\left(bl\right)>P\left(bn\right)$ according to a corpus of English.

This baseline model, however, fails to capture the fact that bn as a sequence is not itself problematic (e.g., subnet, abnormal, abnegate). It is only a problem in a particular position, namely the beginning of the word. One way to address this flaw would be with a trigram model, in which case we would expect $P\left(\#bl\right)>P\left(\#bn\right)$ .Footnote ²⁷ But Vitevitch et al. (Reference Vitevitch, Luce, Charles-Luce and Kemmerer1997) go further to assess the role of position in phonotactic acceptability. They constructed nonce CVC syllables of high and low probability, with probability determined using both transition (i.e., bigram) probabilities and the probabilities of segments in particular positions (initial-medial-final). Bisyllabic forms were then created that varied in stress placement and covered all possible combinations of high- and low-probability syllables. Their task elicited gradience directly by asking participants to rate words on a scale from 1 (Good) to 10 (Bad). They found that stress placement as well as probability played a role in these judgments: words with first-syllable stress were rated higher than forms with second-syllable stress, and words with two high- (low-) probability syllables were rated highest (lowest) overall.

Similarly, the probabilistic parser of Coleman and Pierrehumbert (Reference Coleman, Pierrehumbert and Coleman1997) incorporates the role of prosodic structure via a (non-recursive) context-free grammar in which the non-terminals encode stress, syllable, and positional information. In the example rule in (30), S, O, and R represent syllable, onset, and rhyme, respectively, and the “sf” designation further indicates that these components are stressed and final.

1. (30) $\text{Ssf}\to \text{Osf}\text{Rsf}$

Each word component is assigned a path based on the rules that generate it, and the probability of a path is computed from a parsed training corpus. The probability of an entire word is then the product of its component paths. While the probability of a word’s worst component was found to be significantly correlated with the experimental results of Coleman (Reference Coleman1996), the strongest predictor was the log probability of the entire word (with log probabilities being used to address the effect of word length). In this global and probabilistic approach to word acceptability, the presence of well-formed subword components can mitigate the effect of unattested or infrequent ones.

Another whole-word conception of likelihood is in terms of overall similarity to the existing words in the lexicon. Greenberg and Jenkins (Reference Greenberg and Jenkins1964) define this similarity in terms of the number of substitutions needed to convert a nonword into an actual word, and Ohala and Ohala (Reference Ohala, Ohala, Ohala and Jaeger1986) provide experimental evidence that English speakers are in fact sensitive to varying degrees of similarity among nonwords.Footnote ²⁸ This conception of a word’s similarity has been extended to define its neighborhood density, or the number of existing words it is similar to. For example, blick is one substitution away from a number of existing words such as brick, black, block, slick, click, flick, blip, etcetera. How similar an existing word has to be to count as a “neighbor” can vary, but a common definition is that it requires a single string edit operation, to include additions and deletions as well as substitutions.

Of course, phonotactic probability and neighborhood density are correlated – forms with low- (high-) probability subword components will also have small (large) neighborhoods – making it difficult to tease apart which one is responsible for speakers’ acceptability ratings. The study of Bailey and Hahn (Reference Bailey and Hahn2001), however, aimed to do exactly that. Moving beyond the “single string edit” definition of a neighbor, they instead propose a more sophisticated generalized neighborhood model, in which edit operations are weighted to reflect the phonological distance between two forms (e.g., back differs from both pack and sack by one substitution, but the difference between /b/ and /p/ is smaller than the difference between /b/ and /s/ in terms of shared features). They found that both phonotactic probability and lexical neighborhoods have significant effects on acceptability, and that one is not subsumed by the other. However, given that less than half of the variance in their collected ratings was accounted for with these measures combined, they ultimately conclude that there is still more to understand about these factors in particular and gradient acceptability in general.

When teasing apart the relative influences of phonotactic probability and neighborhood density, Shademan (Reference Shademan, Baumer, Montero and Scanlon2006) argues for a need to consider task effects. In particular, the inclusion of actual words in the set of experimental stimuli (as in Bailey and Hahn Reference Bailey and Hahn2001) may amplify lexical effects, as was previously suggested by Vitevitch and Luce (Reference Vitevitch and Luce1998) in the context of word-processing tasks. Shademan’s own experiments tested words in the four logically possible combinations of high/low probability and high/low lexical similarity. When only nonce words were included, probability had a greater correlation with the acceptability ratings (scored from 1 to 7) than lexical similarity. When both actual and nonce words were included, the effect of lexical similarity became more pronounced, though it still was not as strong as the effect of probability.

Even with their relative roles still in question, phonotactic probability and neighborhood density do not tell the whole story of gradient acceptability. Other work has identified significant contributions of phonological knowledge beyond prosodic structure. Hay et al. (Reference Hay, Pierrehumbert, Beckman, Local, Ogden and Temple2004) tested nonce words containing nasal-obstruent clusters that vary in their frequency (e.g., [nt] is very frequent, [mθ] is unattested, and [nf] is attested but infrequent). Cluster frequency was highly correlated with participants’ acceptability ratings (which were determined on a scale of 1 to 10), at least for the attested clusters. The unattested clusters unexpectedly defied this pattern, being rated higher than some low-probability attested clusters. A follow-up experiment suggested that the additional information speakers might be incorporating into their assessment of nonce word likelihood includes morphological analysis (i.e., parsing unattested clusters as spanning a morpheme boundary) as well as long-distance phonological effects like the OCP.

Another potentially valuable source of information missed by basic n-gram models is the feature-based representations of segments, as in the Hayes and Wilson (Reference Hayes and Wilson2008) phonotactic model (discussed previously in Section 3.2.3). To further explore this potential, Albright (Reference Albright2009) uses the probability of sequences of natural classes in addition to specific segments to estimate the likelihood of an onset cluster that is not present in a training corpus. For example, both [bd] and [bn] are unattested onsets in English, which would be treated similarly by a bigram model. By generalizing over features instead, Albright’s model predicts the preference for [bn] because it has more features in common with attested onsets compared to [bd].

Yet another line of research has instead questioned the initial assumption that gradient acceptability ratings reflect gradient grammatical knowledge. In addition to the explanatory power of phonotactic probability and neighborhood density not being complete, Gorman (Reference Gorman2013) shows that it is not even consistent, as in some cases these gradient models are outperformed by a baseline categorical model that only considers whether a word includes an illicit component. Like Shademan (Reference Shademan, Baumer, Montero and Scanlon2006), Gorman suggests the potential for task effects to play a role in gradient acceptability, including the implications of asking participants to use a scale in the first place (e.g., Armstrong et al. Reference Armstrong, Gleitman and Gleitman1983 show that when given the option to use intermediate ratings, participants will provide gradient judgments on how odd or even a number is, even though these properties are categorical by definition).

Kahng and Durvasula (Reference Kahng and Durvasula2023) also directly challenge the assumption that gradient acceptability reflects an underlying grammar of gradient generalizations, arguing that the perceptual system introduces bias and variance that can influence the cline of acceptability. In their experiment, Korean speakers rated forms with illicit clusters lower than those in which a vowel breaks up the cluster. More surprising, clusters with [c] as the first consonant were rated higher than those with [b] as the first consonant, even though both clusters are unattested. This discrepancy is hard to explain as a gradient generalization. A feature-based approach would actually predict [bC] to be better than [cC], since the former has more features in common with attested sequences (i.e., nasal clusters are attested, so the voicing of [b] should give it an advantage).

In addition to rating acceptability, participants were also asked to identify the medial vowel in the stimuli (with “no vowel” as an option). Factoring that information into the analysis, the authors found that (1) participants rated forms more acceptable when they heard an illusory vowel in the disallowed clusters, and (2) they were more likely to hear illusory vowels in [cC] compared to [bC]. Based on this, they propose a model in which categorical grammatical constraints operate over a probability distribution of perceptual representations. Hearing an illusory vowel in illicit forms results in a perceived licit form that the grammar recognizes as well formed, with the likelihood of hearing these vowels (i.e., the source of gradience) attributed to the perceptual system. They conclude with a larger suggestion that proposals for gradient grammatical generalizations based on acceptability judgments should be supported by an investigation into what participants actually perceived.Footnote ²⁹

How best to account for phonotactic knowledge remains an ongoing question of interest. We will return to research on phonotactics – particularly how they are learned – in Section 7.1. But first, the next two sections will briefly survey the past and current phonological research employing information theory and neural networks, respectively.

5 Information Theory

This section demonstrates the utility and flexibility of information theoretic methods by highlighting examples of their application to a range of problems of phonological interest.Footnote ³⁰

6 Neural Networks

The potential of connectionist models of morpho-phonology has long been recognized (e.g., Rumelhart and McClelland Reference Rumelhart, McClelland, Rumelhart and McClelland1986; Gasser and Lee Reference Gasser and Lee1990), though perhaps underutilized, with current developments sparking renewed interest. Due to space limitations, this section will be necessarily brief, but readers are referred to Alderete and Tupper (Reference Alderete, Tupper, Bosch and Hannahs2018) as well as any of the works cited in what follows for more context on and examples of the use of neural networks for phonological learning, modeling, and theorizing. Pater (Reference Pater2019) in particular provides an accessible introduction to neural networks and a valuable discussion of their history and potential for greater integration with generative linguistics research.Footnote ³³ Taking an even stronger stance on the future, Boersma et al. (Reference Boersma, Benders and Seinhorst2020) argue that only a neural network model will be capable of accounting for the full range of behavioral data associated with the phonetics – phonology interface.

As articulated by Goldsmith (Reference Goldsmith, Lima, Corrigan and Iverson1992a), neural networks offer a means for conducting phonological analysis without committing to typical generative assumptions of a highly structured LAD, as well as a clearer route to drawing connections with other areas of cognitive science. As an example, Goldsmith (Reference Goldsmith and Davis1992b) presents a case study on using a neural network to model stress patterns. The units of the network correspond to the bottom row of a metrical grid (i.e., syllables), with stress corresponding to local maxima (greater activation than neighboring units). Activation may be inherent (determined by syllable position or weight) or derived by lateral inhibition of a neighboring unit. Particular settings of inherent activation and the inhibitory weights simulates effects like alternating stress and avoidance of stress clash. These effects are achieved through iterative recomputation until equilibrium, rather than through an ordered derivation that manipulates hidden structure. Goldsmith and Larson (Reference Goldsmith, Larson, Canakis, Chan and Denton1993) further show how syllabification can be modeled using activation to encode a segment’s level of sonority, where a local maxima this time identifies a syllable’s nucleus. Language-specific constraints or parameters are the result of different weights that control the spread of activation through the network, as well as activation thresholds that determine what counts as a peak.Footnote ³⁴

Given that the potential for these models to provide insight into language acquisition and cognition depends on their interpretability, several researchers have sought evidence of linguistic structure in the patterns of activation of hidden layers. For example, Alishahi et al. (Reference Alishahi, Barking, Chrupała, Levy and Specia2017) present results from experiments showing that phonemes can be recovered from the hidden layer representations of a recurrent neural network (RNN) trained to map pairs of spoken language and images into a semantic space. Success of phoneme recovery was greatest with the early (first and second) hidden layers and then decreased with each successive layer. Similarly, Smith et al. (Reference Smith, O’Hara, Rosen, Smolensky, Ettinger, Pavlich and Prickett2021) test a gestural harmony (Smith Reference Smith2018) account of stepwise vowel harmony with an encoder–decoder model that maps strings of phonological units to sequences of articulator movements. The decoder’s patterns of attention indicated that it was attentive to states corresponding to harmony-triggering vowels throughout the span of the word, consistent with the gestural overlap account of vowel harmony.

A clear advantage of neural networks for modeling acquisition is their ability to work with raw speech data directly, rather than the discrete input representations assumed by the majority of phonological learning models. This potential integration of phonetic and phonological learning is explored with a generative adversarial network (GAN) in Beguš (Reference Beguš, Ettinger, Jarosz and Nelson2020a, Reference Beguš2020b). Generative adversarial networks consist of a generator network tasked with generating data and a discriminator network tasked with determining whether an input is real or generated. The two networks are trained in tandem such that the discriminator’s pattern of errors is used by the generator to improve its ability to generate data (i.e., its ability to fool the discriminator). Beguš (Reference Beguš, Ettinger, Jarosz and Nelson2020a, Reference Beguš2020b) used English data to train a GAN to generate voiceless stop-vowel sequences with and without an initial /s/. In the generated samples, the voice onset time (VOT) of the vowel was significantly shorter in the presence of the initial /s/, as predicted by the English pattern of allophonic variation between aspirated and unaspirated voiceless stops. To again address the question of interpretability and better understand the actual learning mechanisms at work, Beguš proposes a logistic regression–based method for finding correlations between the model’s latent space and output variables such as presence versus absence of /s/.

In other work, neural networks are used to test the necessity of various types of linguistic structure for learning phonological patterns. For example, Doucette (Reference Doucette, Gibson, Linzen, Sayeed, van Schijndel and Schuler2017) showed that phonotactic learning with an RNN is possible without relying on repeated features (i.e., alpha variables), and Mayer and Nelson’s (Reference Mayer, Nelson, Ettinger, Jarosz and Nelson2020) RNN phonotactic learner performed comparably to Hayes and Wilson’s (Reference Hayes and Wilson2008) MaxEnt learner on Finnish vowel harmony, even without the augmentation of a tier. Prickett and Pater (Reference Prickett, Pater, Ettinger, Hunter and Prickett2022) likewise forgo the need for prespecified constraints with an encoder–decoder model that achieved state-of-the-art accuracy on Tesar and Smolensky’s (Reference Tesar and Smolensky2000) stress pattern dataset and correctly generalized 112 of its 124 patterns. Delving further into learning biases, Prickett (Reference Prickett2019) shows that learning in a sequence-to-sequence model can simulate proposed biases for learning process interactions, namely maximal utilization (all rules apply maximally; Kiparsky Reference Kiparsky, Bach and Harms1968) and transparency (interactions are not opaque; Kiparsky Reference Kiparsky and Dingwall1971). And Prickett (Reference Prickett2021) argues that such models also capture formal language theoretic complexity biases of the sort discussed in the next section.

In addition to addressing the challenges of interpretability, future research using neural networks will hopefully shed light on how the choice of architecture and other design options affect what they can learn, as well as how these choices relate to the kinds of grammatical distinctions that linguists make. One approach to understanding neural networks has been to draw on tools from formal language theory (see Merrill Reference Merrill, Drewes and Volkov2023 for an overview), which we turn to in the next section.

7 Formal Language Theory (FLT)

A formal language theoretic approach to phonology emphasizes the formal structure of linguistic patterning in order to identify abstract universal properties, which in practice often relate to computational complexity. This formal structure is recognized by first representing phonological patterns with mathematical objects. For example, a phonotactic constraint like *[−son, +voice]# (i.e., words cannot end with voiced obstruents) can be represented with a set of strings that do not violate it (e.g., {aba, ba, ap, pa, …}) or a function that maps a given string to 0 or 1 depending on whether it violates the constraint (e.g., f(aba) = 0, f(ab) = 1, etc.).Footnote ³⁵ Similarly, a phonological rule can be represented with a function (obligatory rule) or relation (optional rule) that maps an input to an output or set of outputs, respectively (e.g., f(ab) = ap or {ab, ap}). The advantage of representing patterns in this way is the ability to identify their invariant structural properties (examples of which will be discussed in what follows), which hold regardless of the choice of grammatical formalism (rules, constraints, etc.).

Importantly, this emphasis on structure is fully compatible with statistical and quantitative approaches. For example, Hayes and Wilson’s (Reference Hayes and Wilson2008) MaxEnt phonotactic learner (discussed in Section 3.2.3) makes use of a template for constraints in order to structure and narrow the hypothesis space. Formal language theory studies different kinds of templates, identifying what kinds of patterns they can and cannot express and what distinctions algorithms have to make to learn them. These results will be true regardless of whether or not the constraints are weighted or used to derive probability distributions.

Foundational work in this vein includes the aforementioned (Section 3.3.1) independent discovery by Johnson (Reference Johnson1972) and Kaplan and Kay (Reference Kaplan and Kay1994) that SPE grammars are regular provided the rules do not reapply to their own structural changes.Footnote ³⁶ This means an individual phonological rule can be compiled into an FST), and – since the regular relations are closed under composition – an ordered set of rules can likewise be represented with a single FST (i.e., the entire grammar is also regular). This finding established a computational boundary between phonology and other domains like morphology and syntax, which include patterns that are more complex than regular (see Carden Reference Carden1983; Culy Reference Culy1985; Shieber Reference Shieber1985; Kobele Reference Kobele2006; Heinz and Idsardi Reference Heinz and Idsardi2011, Reference Dillon, Dunbar and Idsardi2013). It also indicated that while SPE captures the basic intuition that phonological changes affect sounds in particular contexts, as a grammatical formalism, context-sensitive rules are more powerful than necessary for phonology.

The finite-state modeling of phonology has also been a productive route to the development of software for implementing morpho-phonological systems (Beesley and Karttunen Reference Beesley and Karttunen2003; Hulden Reference Hulden and Kreutel2009; Aksënova Reference Aksënova2020; Gorman and Sproat Reference Gorman and Sproat2021). Koskenniemi’s (Reference Koskenniemi1983) two-level rule approach in particular has served as the basis for morphological analyzers for several languages, including low-resource languages that are less amenable to deep-learning approaches (e.g., Çöltekin Reference Çöltekin, Calzolari, Choukri, Maegaard, Mariani, Odijk, Piperidis, Rosner and Tapias2010 and Washington et al. Reference Washington, Ipasov, Tyers, Calzolari, Choukri, Declerck, Doğan, Maegaard, Mariani, Moreno, Odijk and Piperidis2012).

Regularity then provides a well-defined proposed computational universal that is sufficiently expressive while ruling out a great many non-phonological patterns. However, Heinz (Reference Heinz2011a, Reference Heinz2011b) – the culmination of a line of work initiated in Heinz (Reference Heinz2007, Reference Heinz2009, Reference Heinz2010b) – offers the further hypothesis that phonological patterns are in fact subregular and belong to proper subsets of the regular languages and relations. This hypothesis is motivated by (1) typology, as the regular classes still admit phonologically implausible patterns; and (2) learnability, as the regular classes are not learnable under a variety of settings, including in the limit from positive data (Gold Reference Gold1967) and the probably approximately correct framework (Valiant Reference Valiant1984, Reference Valiant2013).

With respect to typology, the subregular hypothesis offers clear and testable predictions designed to advance our understanding of the nature of phonological computation. For example, Gainor et al. (Reference Gainor, Lai, Heinz, Choi, Hogue, Punske, Tat, Schertz and Trueman2012) and Heinz and Lai (Reference Heinz, Lai, Kornai and Kuhlmann2013) show that progressive and regressive vowel harmony is not just regular but subsequential (i.e., deterministic), and dominant-recessive and stem-controlled harmony patterns are what they call weakly deterministic.Footnote ³⁷ In contrast, unattested patterns like Sour Grapes (Padgett Reference Padgett, Suzuki and Elzinga1995; Wilson Reference Wilson2003) and Majority Rules (Lombardi Reference Lombardi1999; Baković Reference Baković2000) fall outside of these boundaries (the latter is in fact non-regular). In the same vein, Jardine (Reference Jardine2016a) argues that tonal patterns regularly exhibit greater computational complexity compared to segmental ones.

Such hypotheses are informed by the current state of knowledge of what is and is not attested, and therefore serve to highlight what kinds of patterns we should be looking for in order to extend that state of knowledge. In response to Heinz and Lai (Reference Heinz, Lai, Kornai and Kuhlmann2013) and Jardine (Reference Jardine2016a), McCollum et al. (Reference McCollum, Baković, Mai and Meinhardt2020) and Meinhardt et al. (Reference Meinhardt, Mai, Baković and McCollum2024) present vowel harmony patterns that meet Jardine’s (Reference Jardine2016a) definition of unbounded circumambience (= the conditions for a change require unbounded lookahead in both directions from the target) and therefore serve as evidence that challenges his argument.Footnote ³⁸ The latter work further draws a distinction between such patterns and those they call unbounded semiambient (= the conditions for a change require unbounded lookahead in at most one direction from the target).Footnote ³⁹ These kinds of typological investigations thus result in valuable, nuanced characterizations of the computations necessary to recognize or represent a phonological pattern.

With respect to learnability, the fact that the regular languages and relations are not learnable from positive data means the property of regularity does not sufficiently limit the hypothesis space of a phonological learner. In contrast, the subregular properties that delimit proper subsets of regular do enable learning under these conditions. The next two sections review the work demonstrating that potential in the learning of phonotactics and mappings, respectively, much of which builds on computational learning theory (Osherson et al. Reference Osherson, Weinstein and Stob1986; Jain et al. Reference Jain, Osherson, Royer and Sharma1999; Mohri et al. Reference Mohri, Rostamizadeh and Talwalkar2018) and grammatical inference (de la Higuera Reference de la Higuera2010; Heinz and Sempere Reference Heinz and Sempere2016; Wieczorek Reference Wieczorek2017).

8 Conclusion

The variety and volume of work covered in this Element are a testament to how prevalent quantitative and computational approaches to phonology have become, and that trend is likely to not only continue but grow in the years ahead. Researchers focusing on a wide range of puzzles and problems related to the acquisition and representation of phonological knowledge are more fully embracing the value if not the necessity of computational, mathematical, and/or statistical tools in their investigations. In turn, these methods are becoming an increasingly necessary component of the teaching of and training in phonology as a field of study. The kinds of analysis they enable have greatly augmented our capacity for identifying and characterizing phonological patterns and for studying what can be learned under what conditions, from both a formal and an empirical perspective. Lastly, these approaches have equipped phonologists with a range of options for implementing our theories, forcing us to make them more precise and enabling us to better assess what remains to be uncovered with respect to the phonological component of natural languages.