Hostname: page-component-8448b6f56d-m8qmq Total loading time: 0 Render date: 2024-04-18T08:26:37.329Z Has data issue: false hasContentIssue false

Figurative meaning in multimodal work by an autistic artist: a cognitive semantic approach

Published online by Cambridge University Press:  02 June 2020

JENNY HARTMAN*
Affiliation:
Department of Language Studies, Umeå University, Umeå, Sweden Centre for Languages and Literature, Lund University, Lund, Sweden
CARITA PARADIS
Affiliation:
Centre for Languages and Literature, Lund University, Lund, Sweden
*
Address for correspondence: Jenny Hartman, Department of Language Studies, Umeå University, SE-90187 Umeå, Sweden. e-mail: jenny.hartman@umu.se
Rights & Permissions [Opens in a new window]

Abstract

Research on figurative meaning in Autism Spectrum Disorder (ASD) would benefit from considering a greater variety of data types and using more diverse methods. Previous studies have predominantly applied experimental methods to investigate processing of figurative language (mostly metaphor) and have for the most part concluded that individuals with ASD have deficits in figurative language comprehension. In this study, we focus on the creation and communication of figurative meaning in discursively situated and thematically organized verbal, gestural, and pictorial data published by an autistic artist in the form of videos and comics. Across three prominent experiential themes in the data, we isolate types of conceptualizations and generalize over mappings between target and source experiences. We find that the data are rich in figurative meaning expression (e.g., metaphor and metonymy) conveyed through language, co-speech gestures, and pictures in ways that are clearly embodied (experientially based) and that reflect affordances and constraints of these modes of communication. While our case study of meaning production does not contradict previous research on figurative meaning comprehension in populations, it does indicate benefits of taking a broader and multimodal approach to figurative meaning in research on ASD.

Type
Article
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted reuse, distribution, and reproduction in any medium, provided the original work is properly cited.
Copyright
© UK Cognitive Linguistics Association 2020

1. Introduction

Research on figurative meaning in Autism Spectrum Disorder (ASD) has predominantly investigated the processing and comprehension of figurative language through experimental methods, and a great number of studies have concluded that individuals with ASD have difficulties understanding figurative meaning (e.g., Chahboun, Vulchanov, Saldaña, Eshuis, & Vulchanova, Reference Chahboun, Vulchanov, Saldaña, Eshuis and Vulchanova2016, Reference Chahboun, Vulchanov, Saldaña, Eshuis and Vulchanova2017; MacKay & Shaw, Reference MacKay and Shaw2004; Mashal & Kasirer, Reference Mashal and Kasirer2011; Melogno, D’Ardia, Pinto, & Levi, Reference Melogno, D’Ardia, Pinto and Levi2012; Olofson et al., Reference Olofson, Casey, Oluyedun, Van Herwegen, Becerra and Rundblad2014; Rundblad & Annaz, Reference Rundblad and Annaz2010a). The main focus in this line of research has been verbal metaphor. A more limited number of studies have considered other forms of figurative language, such as idioms, irony, and metonymy. We propose that it is not sufficient to determine whether or not, or to what extent, individuals with ASD comprehend figurative language; there is also a need to explore the ways in which they convey figurative meaning in actual language use and through additional modes of communication such as gestures and pictures.

In this paper, we explore experiential meaning conveyed by an autistic artist through a body of work published online (videos and comics). The paper is intended as a contribution to research on figurative meaning in ASD, but also to research on figurative meaning more generally. As Littlemore points out, research on metaphor has emphasized “homogeneity over heterogeneity” and has focused largely on “prototypical” and “normal” experiences and people (Reference Littlemore2019, pp. 48–49, in a discussion of Lakoff & Johnson’s work). She argues that variation should be a focus of metaphor research, because it can tell us more about the variety of “ways in which people perceive the world […], the reasons for these different world views and the mechanisms through which they develop”, while at the same time adding to our knowledge of “the nature of embodied metaphor itself” (Reference Littlemore2019, p. 49). In our study, data are not treated as representative of meaning making by people with ASD. Rather than generalizing from the data onto a very diverse group of individuals, we use the data to make a case for broadening the scope of research on figurative meaning in ASD. A cornerstone of Conceptual Metaphor Theory (CMT) is that metaphors and other kinds of figurative meaning are “primarily a matter of thought and action and only derivatively a matter of language” (Lakoff & Johnson, Reference Lakoff and Johnson1980, p. 153). This assumption requires that our explorations of figurative meaning also account for other forms of communication (Cienki & Müller, Reference Cienki, Müller and Gibbs2008; Forceville, Reference Forceville2002, Reference Forceville and Gibbs2016a; Müller & Cienki, Reference Müller, Cienki, Forceville and Urios-Aparisi2009). Our approach, which investigates meaning production across communicative modes, has the potential to allow new and significant insights into these areas of research and can be of interest to researchers in the language sciences and psychology, as well as gesture and comics studies.

2. Research on figurative meaning and ASD

In this section, we briefly report on previous research on figurative meaning in ASD. Some researchers have identified dilemmas in this line of autism research (e.g., Giora, Gazal, Goldstein, Fein, & Stringaris, Reference Giora, Gazal, Goldstein, Fein and Stringaris2012; Hobson, Reference Hobson2012; Olofson et al., Reference Olofson, Casey, Oluyedun, Van Herwegen, Becerra and Rundblad2014; Rundblad, Reference Rundblad, Semino and Demjén2017), and, much like these scholars, we perceive some theoretical and methodological matters in need of consideration.

2.1. theoretical scope

A central dichotomy in research on ASD is that between figurative and literal language, and in different ways, this distinction underpins the design of many studies. Some of these are designed to compare the comprehension of literal and metaphorical expressions (e.g., Chahboun et al., Reference Chahboun, Vulchanov, Saldaña, Eshuis and Vulchanova2017; Kasirer & Mashal, Reference Kasirer and Mashal2014), while other studies incorporate literal expressions as alternative answers in tasks meant to test metaphor comprehension (e.g., Olofson et al., Reference Olofson, Casey, Oluyedun, Van Herwegen, Becerra and Rundblad2014). With some exceptions (e.g., Giora et al., Reference Giora, Gazal, Goldstein, Fein and Stringaris2012), the standing of these notions is not discussed at any length. The results of our study indicate that a separation between figurative and literal meaning needs to be considered in less absolute terms and from the perspective of diverse modes of communication.

Most studies of figurative language in ASD have focused on metaphor. Some studies have considered idioms (e.g., Chahboun et al., Reference Chahboun, Vulchanov, Saldaña, Eshuis and Vulchanova2016; Mashal & Kasirer, Reference Mashal and Kasirer2011, Reference Mashal and Kasirer2012), a few have addressed metonymy (Rundblad & Annaz, Reference Rundblad and Annaz2010a, Reference Rundblad and Annaz2010b; Van Herwegen & Rundblad, Reference Van Herwegen and Rundblad2018), the odd one a greater variety of figurative expressions (e.g., MacKay & Shaw, Reference MacKay and Shaw2004). In light of the abundance of metaphor studies, some researchers have pointed out that there is little agreement in the field on the definition of metaphor (Olofson et al., Reference Olofson, Casey, Oluyedun, Van Herwegen, Becerra and Rundblad2014; Rundblad, Reference Rundblad, Semino and Demjén2017; Rundblad & Annaz, Reference Rundblad and Annaz2010b). The lack of a robust definition of metaphor is arguably at times a by-product of a more fundamental problem – the absence of a theoretical framework for meaning making more broadly (see Hobson, Reference Hobson2012, for some discussion) – something that directly impacts experimental design and hence results. In this paper, we take a broad approach to figurative meaning that comprises a range of processes, including but not limited to metaphorization.

2.2. data types

Research on figurative meaning in ASD has focused almost exclusively on language, despite the fact that a multimodal approach to figurative meaning may be especially appropriate in autism research. A limited number of studies have incorporated pictorial expression of figurative meaning in ASD, primarily as visual support for linguistic stimuli (Chahboun et al., Reference Chahboun, Vulchanov, Saldaña, Eshuis and Vulchanova2016; MacKay & Shaw, Reference MacKay and Shaw2004; Mashal & Kasirer, Reference Mashal and Kasirer2012; Olofson et al., Reference Olofson, Casey, Oluyedun, Van Herwegen, Becerra and Rundblad2014). Individuals with ASD have been described as having a visual “cognitive style” (Kamio & Toichi, Reference Kamio and Toichi2000, p. 865) or as being prone to “visual thinking” (Kana, Keller, Cherkassky, Minshew, & Just, Reference Kana, Keller, Cherkassky, Minshew and Just2006, pp. 2488–2499). Dr Temple Grandin, an influential autistic author and scientist, writes, “I think in pictures. Words are like a second language to me” (Reference Grandin1995, p. 19). The speaker in our data says that it is easier to convey her experiences visually than verbally (HB, Empathy). Against the background of such research claims and experiential accounts, it is of significance to consider the role of pictures in mediation of figurative meaning by individuals with ASD.

Gestures have also been overlooked in research on figurative meaning in ASD, even though difficulties using and understanding gestures have long been considered to be central to ASD. Such difficulties are, in fact, part of the diagnostic criteria for the disorder (DSM-5, American Psychiatric Association, 2013; de Marchena et al., Reference de Marchena, Kim, Bagdasarov, Parish-Morris, Maddox, Brodkin and Schultz2019; Eigsti & de Marchena, Reference Eigsti and de Marchena2017). Littlemore argues that more studies are needed of embodied metaphor production by individuals with ASD and, moreover, that such studies require “a strong focus on the use of gesture” (Reference Littlemore2019, p. 175). To our knowledge, no studies have focused specifically on the use of gestures for the expression of figurative meaning by individuals with ASD. There are also, more generally, gaps in research on the use of gestures in ASD; de Marchena et al. point out that “gestures that are used during the course of developed, fluent speech are relatively understudied in this population” and that “co-speech gestures have never been examined in verbally fluent autistic adults” (Reference de Marchena, Kim, Bagdasarov, Parish-Morris, Maddox, Brodkin and Schultz2019, p. 1439, italics in original).

A multimodal approach is also pertinent in research on ASD because it has been suggested that at the core of ASD are problems with integration across modalities (de Marchena & Eigsti, Reference de Marchena and Eigsti2010; Eigsti, Reference Eigsti2013; Kana et al., Reference Kana, Keller, Cherkassky, Minshew and Just2006). In fact, it has been proposed that “integrating across modalities in both production and comprehension may be one [of] the greatest communication challenges faced by autistic people” (de Marchena et al., Reference de Marchena, Kim, Bagdasarov, Parish-Morris, Maddox, Brodkin and Schultz2019, p. 1451, italics in original). Multimodal studies that take a usage-based approach by considering authentic communication, in line with what we propose, can potentially offer valuable complementary insights to, for example, neuroimaging studies and cognitive testing.

2.3. methods

Research on figurative meaning in ASD has been primarily experimental with a focus on figurative language processing and has predominantly operationalized such processing as a function of response times and response accuracy, and most studies have concluded that individuals with ASD have deficits tied to figurative meaning comprehension (via language). Varied explanations for these deficits have been proposed: general language and/or communicative deficits, reduced ability to infer others’ intentions; Theory of Mind deficits; impaired executive functions; and differences in language processing (for overviews/reviews, see Chahboun et al., Reference Chahboun, Vulchanov, Saldaña, Eshuis and Vulchanova2017; Eigsti, de Marchena, Schuh, & Kelley, Reference Eigsti, de Marchena, Schuh and Kelley2011; Kalandadze, Norbury, Nærland, & Næss, Reference Kalandadze, Norbury, Nærland and Næss2018; Vulchanova, Saldaña, Chahboun, & Vulchanov, Reference Vulchanova, Saldaña, Chahboun and Vulchanov2015).

While the research methods vary, what many experimental studies have in common is a focus on the processing and comprehension of isolated words or short sentences embedded in restricted contexts. Experiments have advantages in answering many types of research questions, but there are also problems associated with using artificial stimuli to assess language comprehension. Zwaan argues that using “decontextualized words and sentences as stimuli […] is problematic because the resulting patterns of brain and behavioral responses that are obtained in experiments cannot simply be extrapolated to discourse comprehension” (Reference Zwaan2014, p. 230). It is important to consider how the results of experimental studies further our understanding of actual figurative meaning use in ASD, which is contextually embedded (discursively, socially, and culturally), grounded in sensorimotor experience, and involves dynamic interplay between comprehension and production.

3. Analytical framework

In this study we focus on figurative meaning as it is achieved through what we refer to as comparative and salience construals (Paradis, Reference Paradis2005). Comparative construal is realized in communication as a variety of meaning phenomena, such as metaphor, analogy, simile, and literal comparison, and can be expressed via language, gestures, and pictures. The basis for any kind of comparative construal is a mapping between conceptual domains. As an example, through the conceptual metaphor life is a journey, a relatively abstract target, life, is understood in terms of a journey (source). The mapping between a life and a journey is characterized by construed similarities. Much like a journey, a life is understood as involving a beginning boundary, a spatio-temporally extended path, and an end boundary. As such, this mapping affords structure and meaning to expressions such as Follow your own path and His life was a flaming wreckage. In addition to structural correspondences (boundary, path), expressions that draw upon this mapping can also highlight other shared features such as sensory or emotive correspondences (Hartman & Paradis, Reference Hartman and Paradis2018; Lakoff, Reference Lakoff1990).

Salience construal, on the other hand, does not involve such cross-domain mapping, but instead entails foregrounding of elements within a single conceptual structure. Whereas comparative construal is based on construed similarity between domains, salience construal is a reference-point phenomenon whereby one element provides access to a conceptually contiguous element. Salience construal can be realized through metonymization, zone activation, or facetization (see Paradis, Reference Paradis2004, for a detailed account). In our discussion, we consider metonymization and zone activation. We use metonymization to describe expressions in which one meaning is used to evoke another meaning and these meanings “represent two distinct senses that, out of context, […] are associated with two different lexical items” (Paradis, Reference Paradis2004, p. 252). For instance, through metonymization a part can be used to stand for a whole, as in I need an extra pair of hands (hands for person/part for whole). Zone activation, on the other hand, operates within senses of expressions in communication. Zone activation is also significant to our discussion because different modes of communication – language, gestures, and pictures – come with affordances and constraints that influence the manner in which a meaning is expressed. For instance, we can use language to express the property generous as in She is a generous person. To express this meaning gesturally, we can hone in on a number of aspects of this property concept (generous). We can, for instance, use a Palm-Up-Open-Hand gesture, for which the “prototypical meaning […] can be described as giving, receiving, and showing something on the open hand” (Müller, Reference Müller2017b, p. 292) (hand movement/action for property). These differences between metonymization and zone activation aside, in both cases, the evoked meanings are conceptually contingent to the expressed meanings through experience or convention.

Zone activation directs focus to contextually relevant aspects of a concept’s use potential. As such, this salience operation is also necessary in metaphorization (Paradis, Reference Paradis, Benczes, Barcelona and de Mendoza Ibáñez2011), so that salience and comparative construals can operate jointly. Language, co-speech gestures, and pictures can all serve to facilitate metaphorization by specifying and reinforcing appropriate aspects of an expression’s meaning potential. The metaphorical expression John is a bear can be interpreted in different ways depending on which aspects of bear are intended (e.g., strong, big, huggable, sleepy in the winter, or fond of berries). A gestural approximation of a hug concurrently with the verbal expression can be used to indicate that huggable is the most relevant meaning, in which case a metaphorical verbal expression evokes figurative meaning jointly with a zone activating gesture.

In line with the cognitive semantic model Lexical Meaning as Ontologies and Construal (LOC, Paradis, Reference Paradis2005), we distinguish between conceptually rich meaning domains drawn upon in meaning making (e.g., space or bear) and the schematic and conceptually simpler templates used to structure these domains (e.g., boundary, force). These templates (configurations in LOC) are akin to image schemas (Johnson, Reference Johnson1987; Lakoff, Reference Lakoff1987; in gestures Cienki Reference Cienki and Hampe2005; Mittelberg, Reference Mittelberg2018; in pictures, Forceville, Reference Forceville and Cohn2016b) in that they can be characterized as embodied cognitive structures that capture fundamental perceptual experiences of the world, while also scaffolding new experiences and facilitating the creation of meaning.

4. Data and method

The data were drawn from published work by an autistic artist (HB) and consist of language (spoken and written), gestures, and pictures. Surveying all data types (Table 1), we isolated three central themes in HB’s work: (i) auditory processing, (ii) dissociative experiences, and (iii) echolalia and selective mutism. The expression of experiential meaning associated with these themes was then analyzed in detail.

table 1. Overview of data

Dissociation, depersonalization, lucid dreaming, echolalia, and selective mutism are terms used by HB. The same terminology is employed in research, but we use these terms because they appear in HB’s work. We give very brief descriptions of the phenomena in the respective sections, but avoid lengthy explanations because we are interested in HB’s descriptions rather than accounts of these phenomena from research or clinical perspectives.

4.1. analytical steps

We used the operational questions in Table 2 to guide our analyses of figurative meaning. In the discussion, we sometimes use standard descriptions of mappings (e.g., sounds are objects), but we would like to emphasize that such descriptions do not capture the full complexity of expressed meanings. We agree with Forceville and Paling, who argue that the “a is b formula downplays the dynamic nature of metaphors” (Reference Forceville and Paling2018, p. 4), and that “[w]hat matters in metaphor is that things people do to, or experience through, the source domain are mapped onto the target domain, including the emotions, attitudes and values associated with these things” (Reference Forceville and Paling2018, p. 4; also Gibbs, Reference Gibbs2019; Müller & Cienki, Reference Müller, Cienki, Forceville and Urios-Aparisi2009).

table 2. Questions for consideration of figurative meaning in verbal, gestural, and pictorial communication

4.2. data

Spoken language and co-speech gestures were accessed via a vlog published on Youtube.com (Table 1). Spoken accounts were transcribed and annotated for co-speech gestures. As mentioned, we approach figurative meaning from a production perspective, and consider how it is used in the mediation of the selected target experiences. Accordingly, our view of language and pictures, as well as gestures, is geared towards production. Naturally, co-speech gestures play a role in the comprehension of speech, but more significantly to the aims of this study, we assume that gestures “reflect internal cognitive processes” and that they serve not only “inter-cognitive or communicative functions,” but also “intra-cognitive functions” (Pouw, de Nooijer, van Gog, Zwaan, & Paas, Reference Pouw, de Nooijer, van Gog, Zwaan and Paas2014, p. 1; also Littlemore, Reference Littlemore2019, pp. 65–69). As such, the study of gestures can yield insight into “deeply embodied aspects of the human mind” (Mittelberg, Reference Mittelberg2018, p. 2). In line with Müller and Cienki, we limit our consideration of co-speech gestures to visible “positions, orientations, and movements of the hands and forearms” (2009, p. 301). We do, however, at times comment on additional bodily expressions such as posture and facial expressions. Speech complemented by gestures is underlined in our examples, and speech accompanying the main strokes of gestures is placed in brackets. At times, we describe the placement of gestures in the speaker’s gesture space following McNeill (Reference McNeill1992, p. 89) in terms of the speaker’s right/center/left, upper/center/lower, center/periphery/extreme periphery. In our examples, G, as in G1, G2, …, stands for gesture. We include arrows in the image stills of gestures to indicate their directions and present abbreviated titles of the videos and time codes so that readers can consult the original videos (e.g., Hearing, 1:43; see online supplementary materials, available at <http://doi.org/10.1017/langcog.2020.20> for full titles).

Written language and pictures in comics were accessed via a blog published online (Table 1). The comics have titles that help establish the relevant experiences (e.g., MELTDOWNS!!). Forceville points out that “there are ways of expressing metaphors available to comics […] that are not available to language” (Reference Forceville2017, pp. 275–278; also El Refaie, Reference El Refaie, Forceville and Urios-Aparisi2009, Reference El Refaie2019; Cienki, Reference Cienki and Hampe2005, for gestures). He argues that comics make good data for the study of emotion metaphor because “unlike, for instance, realistic photographs and live-action films, which more or less ‘naturally’ mirror real-life manifestations of emotions, comics and cartoons make use of stereotypical exaggerations and of a rudimentary ‘sign-system’ very much like a language” (Forceville, Reference Forceville2005, p. 71). The comics in our data juxtapose written language and pictures, each medium bringing its own affordances and constraints to the mix.

4.3. ethical considerations and limitations

Using data published online has ethical implications, not least when the creator uses her real name and image. Even when data are published with unrestricted access in the public domain, there are grounds for caution. For this reason, we have secured HB’s permission to include her work in this paper. We would nonetheless like to briefly comment on ethics. Some researchers feel that the identities of bloggers should be protected, not least when they share personal information. Other researchers “argue that it is an issue of document copyright and intellectual property, and thus, bloggers and their material must be cited properly” (Kurtz et al., Reference Kurtz, Trainer, Beresford, Wutich and Brewis2017, p. 8). As Kurtz et al. point out, “while it may be tempting for researchers to anonymize authors when sharing results based on the highly personal stories laid out in blogs, attempts to protect bloggers through anonymization also carry an ethical risk” (2017, p. 3) because ultimately use of such data without proper citation may infringe on the bloggers’ copyright to their materials. We uphold HB’s copyright and refer to her work.

One limitation of our study is that it considers work by a single individual. However, we do not claim that our data are representative of figurative meaning use by autistic persons. Generalization to all or most autistic persons, if/when possible, is not the only route to extending our knowledge of figurative meaning use in ASD; there is also value in testing the limits of variation by considering patterns and idiosyncrasies in actual meaning making. Another potential limitation of our study is the “‘monologue’ format” of our data, wherein the speaker expresses herself “uninterrupted” and not in conversation with others (de Marchena & Eigsti, Reference de Marchena and Eigsti2010, p. 319), although there are asynchronous interactive features of the data, for instance when HB responds to listeners’ questions. There are additional facets of communication of potential interest to figurative meaning in ASD that we do not address, such as gestural timing (Eigsti & de Marchena, Reference Eigsti and de Marchena2017), stress, and intonation (Cienki, Reference Cienki, Cienki and Müller2008).

5. Autism experiences in videos and comics

The following sections account for HB’s verbal, gestural, and pictorial mediation of experiences associated with ASD. There is a wealth of data to consider in her work, but we focus on figurative meaning tied to the three prominent experiential themes: auditory processing (Section 5.1), dissociative experiences (Section 5.2), and echolalia and selective mutism (Section 5.3).

5.1. auditory processing

Auditory processing is a central theme in HB’s videos and comics. She describes difficulties in isolating and identifying speech sounds so that, even when the volume is high, she has trouble interpreting speech. Verbally and gesturally she establishes “active cross-domain mapping[s]” appealing to “the cognitive process of understanding something in terms of something else” (Cienki & Müller, Reference Cienki, Müller and Gibbs2008, p. 486), so that auditory control is accounted for by means of spatial and manual control, as in example (1), wherein she describes why she prefers to have subtitles on when she watches TV.

As demonstrated by example (1) and Figures 1a and 1b, HB verbally and gesturally portrays sounds as allowing physical manipulation, thereby realizing a metaphorical mapping from the spatial domain onto the auditory domain. In gestural expression, the resulting metaphor – sounds are objects – necessitates an initial contiguity relation whereby a “gesture is interpreted metonymically to mean that there is an object [in] the hand, and then, through metaphor, that object is interpreted as a word …” (Mittelberg & Waugh, Reference Mittelberg, Waugh, Forceville and Urios-Aparisi2009, p. 339). HB speaks of grabbing and anchoring sounds, speech sounds in particular, while gesturally enacting such grabbing and anchoring, as shown in Figures 1a and 1b. Concurrently with grabs the words out of the people’s mouths she makes three grabbing gestures with each hand in the extreme periphery of her gesture space at grabs (left hand), words (right hand), out of (both hands), and mouths (both hands). At anchors, she moves both hands (fingers slightly bent and palms first facing the camera and then facing down) from her upper left and right extreme periphery to the center of her gesture space (Figure 1b). She then holds the gesture through … them down for me. Anchors them down for me realizes a metaphor of control as being on the ground (Lakoff Reference Lakoff1999, p. 275), and anchoring is used throughout HB’s work in language, gestures, and pictures to describe various forms of experiential control (e.g., auditory, tactile, emotional).

Fig. 1a. [grabs the words …]

Fig. 1b. … and [anchors] them down for me

In example (1), HB continues by pointing out that, without subtitles, the words sort of distort (she spreads all her fingers on both hands at distort to gesturally enact change) on the way to [her] ear. Verbally, she describes this auditory event by means of a “conduit” metaphor based on a sourcepathgoal schema (Reddy, Reference Reddy and Ortony1979, for the conduit metaphor; Mittelberg, Reference Mittelberg2018, for sourcepathgoal in gesture; Cienki & Müller, Reference Cienki, Müller and Gibbs2008, for conduit in gesture). This schema is realized through language in example (1) by the people’s mouths (source), on the way to (path), and my ear (goal). The goal in this instance is only in part to be understood as HB’s ear; it is perhaps more significantly construed as the perception of sound, represented verbally and gesturally as spatial anchoring. The same metaphor is enacted gesturally, as shown in Figures 1c and 1d, when HB moves her open right hand from next to the camera (Figure 1c) to her ear (Figure 1d) along the sagittal axis.

Fig. 1c. on the way

Fig. 1d. [to my ear]

Through metaphorization, HB verbally and gesturally constructs an overriding mapping – sounds are objects – which in turn allows sounds to be construed as spatially positioned and subject to manipulation (e.g., contained and moved) as in expressions of grab and anchor. Auditory perception is metaphorically construed in terms of a conduit (sourcepathgoal), and auditory control is portrayed as spatial and manual containment.

HB also describes excessive volume as being problematic. She says that when she is out in public … like spinal tap everything is turned up to 11. This is Spinal Tap is a mockumentary from 1984 about a metal band wherein one of the band members proudly states that his amplifier goes to 11, and not 10 like most amps, and is therefore louder. The reference to Spinal Tap, signaled by like, evokes a humorous description of volume that HB uses on a number of occasions. While loud noises are difficult, HB mentions that she can tolerate such sounds better if she is able to control them. She likes toys that talk and make sounds and says that …

Figure 2a exemplifies a figurative gesture accompanying language used non-figuratively (metonymy in gesture, Mittelberg & Waugh, Reference Mittelberg, Waugh, Forceville and Urios-Aparisi2009; metonymy and embodiment, Littlemore, Reference Littlemore and Hampe2017).

Fig. 2a. they’re [slightly too] loud

In the gesture shown in Figure 2a, HB pinches her right thumb and index finger and makes a very slight turning movement, as if carefully turning a small volume dial in the air. The pinching reinforces slightly, and in tandem with the turning motion the combined gesture enacts slightly too loud. The gesture is an instance of metonymization because the action of turning a dial is used to refer to a resulting sound quality (action for property). Directly after the gesture shown in Figure 2a, HB makes another gesture concurrently with the spoken I’m controlling the noise. She stretches her right hand out in front of her at chest height and moves it back and forth in a swiveling motion while also repeatedly moving her thumb to and from the palm of her hand as if pressing a button to enact controlled noise. Like the previous gesture, the gesture shown in Figure 2b effects a salience construal, in this case zone activation – a gesture representing an action (moving the hand as if using a remote control) is used to evoke this action (manual gesture for concomitant action). Much like the gestures shown in Figures 1a and 1b that enact the grabbing and anchoring of sounds, the gestures in Figures 2a and 2b require an initial contiguity interpretation to establish (figuratively) “that there is an object [in] the hand” (Mittelberg & Waugh, Reference Mittelberg, Waugh, Forceville and Urios-Aparisi2009, p. 339). But, unlike the gestures in Figures 1a and 1b, the gestures in Figures 2a and 2b do not entail an additional metaphorical mapping between the imagined object and an auditory perception.

Fig. 2b. [I’m controlling the noise]

HB also accounts for auditory processing in her comics, as in No. 112, Auditory Processing Disorder (Figure 3). The title tells us what the comic is about, and a problematic discrepancy between a spoken message and how it is heard is likewise presented in writing: You speak butterflies. I hear bats.

Fig. 3. Comic No. 112: Auditory Processing Disorder

As in her spoken and gestural accounts of hearing, HB employs a conduit metaphor (sourcepathgoal) in Comic No. 112 to present her problems deciphering speech. The orientation of the schema differs between gestures and pictures due to the constraints and affordances of these modes of communication. The sagittal axis (front–back) does not exist in comic strips, which instead employ a path from left to right. The communicative event in Figure 3 is represented with the speaker to the left and the listener to the right (temporal progression is left to right orientation). The pictorial elements of Figure 3 offer much in addition to what is provided by language. Degree of detail in rendering is an affordance of drawing parallel to linguistic specificity (e.g., a person > a man > James). In Figure 3, selective salience is afforded to parts of the panel through relative degree of detail in visual representation. The speaker (source) is schematic, drawn as an outline to represent anyone, while the listener (goal) is specific (a drawing of HB) and represented with a perplexed facial expression in addition to a question mark in a thought bubble (top right-hand corner of the panel). The spoken message is portrayed as a metamorphosis of a butterfly into a bat along a left-to-right going path. A perplexing distortion of speech is conveyed using affordances of the comic, realizing a construal of sound as visible, animate, and moving from speaker to listener. Transformation of the spoken message is conveyed pictorially through animal metamorphosis and verbally through an inherent contrast between butterflies and bats.

In her comics, HB represents emotion through what Forceville refers to as “indexical signs”, which are aspects of drawings that represent prominent “symptoms” of an emotion, such as facial expressions and body posture or movement (Reference Forceville2005, p. 77). Through zone activation, she thus expresses emotions and sensations via pictorial renditions of “physiological and expressive responses” (Kövecses, Reference Kövecses2000, pp. 133–134) to subjective experiences. As we will see in upcoming sections, she also makes use of pictorial runes (as used by Forceville, Reference Forceville2005, based on Kennedy, Reference Kennedy1982), such as jagged lines to represent loud noise or anger (see Shinohara & Matsunaka, Reference Shinohara, Matsunaka, Forceville and Urios-Aparisi2009, for uses in Japanese manga), or curved lines around a body to represent movement or instability. Pictorial runes are considered metonymical (in our terminology zone activating) because they reflect salient aspects of embodied experience. In addition, “sound magnitude” is sometimes represented by “letter scale”, instantiating multimodal metaphor (Potsch & Williams, Reference Potsch, Williams and Bramlett2012, p. 18).

5.2. dissociative experiences

Dissociative experiences include feelings of derealization and depersonalization, including experienced detachment from one’s own emotions and sensations, as well as more “benign” experiences of being so absorbed in an activity that “the current surroundings are—temporarily—dissociated from awareness” (de Ruiter, Elzinga, & Phaf, Reference de Ruiter, Elzinga and Phaf2006, p. 116). HB describes different forms of such reality distancing in her work. She primarily uses the terms dissociation, depersonalization, and lucid dreaming to describe these experiences, but her uses of these terms are not entirely consistent vis-à-vis different types of experiences. In the following, our use of lucid dreaming (in line with HB’s use of this term) corresponds roughly with the “benign” form of absorption described by de Ruiter et al. (Reference de Ruiter, Elzinga and Phaf2006), while our use of dissociation accounts for more severe and unpleasant sensations of depersonalization and/or derealization.

HB’s descriptions of dissociation feature verbal, gestural, and pictorial representations of unstable motion. Such motion construal foregrounds physical detachment from a source associated with veracity such as reality, the self, or the ground.

HB’s uses of drifting away from reality/myself in Comic No. 42, Anchor Points (Figure 4) evoke the construal of a split self (Lakoff, Reference Lakoff, Fauconnier and Sweetser1996), realizing a metaphor of a person out of control is a divided self (Kövecses, Reference Kövecses2000, p. 44). Alongside language, the drawings in the comic likewise convey this kind of split. There are two figures in the first panel of Comic No. 42 (both are representations of HB) and one is shown to be drifting up and away from the other. Unstable motion is conveyed through language (wibbly wobbly) and pictorial runes in the form of curved lines around the body indicating movement and instability. The simile It’s like solidity disappears evokes a physical basis for understanding lack of experiential stability and permanence (solidity vs. separation). Ropes are visible at the bottom of the second panel under the drifting figure, and in the third panel these ropes attach HB’s legs to her anchor points. Like elsewhere in HB’s work, anchoring counteracts drifting, so that forces tied to physical control convey means of experiential control, which is metaphorically construed as being on the ground (Lakoff Reference Lakoff1999, p. 275).

Fig. 4. Comic No. 42: Anchor Points

In one of her videos (Anxiety), HB describes the occasion when she first experienced depersonalization. She was in a dark pub with family …

HB moves both hands (palms toward her face) down from her eyes at my vision (Figure 5a), and then at and (G2 in example 3) she begins to move her bent arms up and down (alternating sides) as in Figure 5b to, concurrently with I feel all floaty, express the metaphor emotional instability is physical instability. Later in the same video, HB again describes anxiety and depersonalization as feeling floaty. She says she may need some weighted stuff to wear because when she is anxious her body seems to float (Anxiety, 12:04). floating thus represents a sensation that is physically experienced as unstable and that can be counteracted by weighing the body down,Footnote 1 suggesting that HB’s use of this metaphor is thoroughly grounded in her bodily experiences. This embodiment of construal also extends to other motion and force-based metaphors in her work, such as drifting and anchoring.

Fig. 5a. [my vision]

Fig. 5b. [and I feel all floaty]

In another video, HB describes dissociation as a startling experience of a mother figure leaving her alone and incomplete:

As in Comic No. 42 (Anchor Points, Figure 4), a split-self construal is evoked in example (4). HB uses a simile (it’s like there’s a mother figure inside me …) to introduce an extended comparison between dissociation and being left alone (see also the videos Innocent, 17:31 and Mental health, 13:43). One gesture accompanies example (4), in which HB strokes her head with the palms of both hands as she says somewhere in the brain, referring to the location of the mother figure. In reference to the mother figure, HB alternates between using they (they go away) and I (I come back), thus destabilizing the deictic center and reinforcing an ambivalent construal of the self. Additional instances of like extend the simile (it’s like … then I’m like …) and a number of opposites evoke experiential contrast (deictic motion: go away / come back; partwhole: on my own / complete).

Comic No. 120 (Little Miss ExistentialisT, Figure 6) likewise (verbally and pictorially) conveys a split-self construal. In the third panel of the comic, two representations of HB coexist – one observes the other through a magnifying glass. HB writes that she can look outside of [herself] and be objective. Similar to example (4), Comic No. 120 portrays ambivalence with regard to the self. Under the two drawings of HB in the third panel, she writes that she has two states equally – one more rational and objective and another more irrational and paranoid.

Fig. 6. Comic No. 120: Little Miss ExistentialisT

In Comic No. 120, HB verbally portrays the body as a container through expressions such as being inside my head, internalized all these thoughts and feelings, and my brain is full of magical thinking (body as container, Lakoff, Reference Lakoff1987, p. 383; also Kövecses, Reference Kövecses2000). She describes how in [her] teens, she fell inwards into [her] alien delusions and conspiracy theories. Pictorially, this experience of inward motion is shown as a spiral shape, representing a vortex, with a brain at the center and a drawing of HB falling towards it. This multimodal representation also shows words being sucked into the vortex.

HB at times makes use of specific affordances of comics to pictorially express such “inside–outside relations” (El Refaie, Reference El Refaie2019, p. 9). In Comic No. 26, Squash (Figure 7), she describes the experience of bad feelings that [she doesn’t] understand. As in her accounts of dissociation, HB uses uncontrolled motion (drift about) to describe lack of experiential control.

Fig. 7. Comic No. 26: Squash

Emotive control is described in Comic No. 26 through language and pictures in terms of containment. In the first panel, a jagged shape protrudes out of the panel, and in the second panel, HB’s feelings are shown to drift about like clouds, which are not contained within the panel, but go outside of it. In the fourth and last panel, HB’s hands are represented as firmly gripping the clouds, now contained within the panel (emotive control is manual containment). As she gains control in the fourth panel, the pictorial perspective also changes to that of the first-person, thereby allowing the viewer to visually adopt HB’s perspective, potentially facilitating emotive/sensory simulation of her experience.

In her work, alongside descriptions of dissociation, HB describes lucid dreaming, a more pleasant form of reality distancing that involves total absorption in daily activities. In Comic No. 44, The fantasy zone is my Everyday (Figure 8), HB describes how she disconnects from reality and drift[s] away to imagination land,Footnote 2 especially when making coffee, taking [her] antidepressant and doing bathroom activities.

Fig. 8. Comic No. 44: The fantasy zone is my Everyday

Like dissociation, lucid dreaming is portrayed in HB’s work in terms of motion. However, the motion schema is realized differently for dissociation and lucid dreaming. Dissociation is described as movement away from a prominent reference point (drifting away from reality), while lucid dreaming is more often described as movement toward something: drifting off into my creative and fantasy world (Pegasus, 8:17) and drifting away to imagination land (Comic No. 44 / Figure 8). These two types of reality distancing thus foreground different parts of a sourcepathgoal schema: dissociation foregrounds source and path, and lucid dreaming path and goal. However, both forms of reality distancing are accounted for in terms of a split-self and a disconnect from reality (Figure 8).

Through language, gestures, and pictures, HB portrays dissociative experiences in terms of physical experiences. Reality is described in terms of solidity, permanence, and visibility. Dissociation is described as uncontrolled motion away from the ground or the self, and lucid dreaming is portrayed as movement toward an imaginary space. Experiential control, on the other hand, is described in terms of physical control, often as manual containment or forceful downward motion (anchoring). As in her descriptions of sensory and emotive experiences generally, contrast between experiential control and lack of control is mediated as tension between spatial motion / instability and rest/stability.

5.3. echolalia and selective mutism

Echolalia is characterized by repetitions of sounds, including but not limited to speech sounds, and can be considered “among the most recognizable characteristics of autism spectrum disorders” (Stiegler, Reference Stiegler2015, p. 750; see also Bogdashina, Reference Bogdashina2005, pp. 174–195, for communicative and non-communicative echolalia). Echolalia is mediated in HB’s work through construal of the body as a container with an inside and an outside; the inside is private and the outside is public; thoughts are kept on the inside whereas movements and sounds are forcefully moved outwards (for a similar construal in Tourette Syndrome, see Hartman, Reference Hartman2017). In example (5), HB describes how her echolalia is projected outwards; it is not focused inwards. She uses a simile (a bit like) to compare aspects of her echolalia (talking to the invisible people) to one of her favorite TV-series, Dr Who. She explains how her echolalia involves something akin to breaking the fourth-wall. Footnote 3

Construed opposition between an inside and an outside is reinforced in example (5) by the verbs focus (constraining stabilizing action) and project (forceful outward motion). HB uses basically as a way of adjusting the comparative mapping between her echolalia and Dr Who’s fourth wall breaking. Such figurative “tuning” appears throughout HB’s work in conjunction with such mappings (for tuning devices, see Deignan et al., Reference Deignan, Littlemore and Semino2013). In example (5), co-speech gestures accompany both project outwards (G1, G3) and focus inwards (G2). She gesturally enacts project it outwards (twice as shown in Figures 9a and 9c) by indicating a forceful outward trajectory of her echolalia (force in gesture, Mittelberg, Reference Mittelberg2018). For both these gestures, HB moves her open hands in an arc from her shoulders to the left, away from her body. At focus inwards, on the other hand, she moves her slightly cupped hands close to the head front to back, as shown in Figure 9b.

Fig. 9a. … sort of [projects it outwards] …

Fig. 9b. … rather than [focus inwards] …

Fig. 9c. … I [project it outwards]

In other words, verbally and gesturally, construal of the body as a container is combined with forceful deictic motion. A boundary between inside and outside, described in example (5) as the fourth wall, can be traversed through echolalia.

HB describes her echolalia as being of two kinds. Alongside speech, her gestures indicate forceful and rhythmical qualities of these types of echolalia, making the descriptions accessible also to viewers who are unfamiliar with her spoken references (Family Guy, the Kool-Aid Man). The first type of echolalia is described as:

Along with example (6), HB looks to the left (oh no), to the right (oh no), and then she leans in close to the camera for an emphatic OH YEAH. She goes on to compare this to a second type of echolalia:

The speech in examples (6) and (7) and the concurrent gestures in Figures 10a and 10b illustrate both salience and comparative construal. The two types of echolalia are comparatively (metaphorically) construed as a steady train and the kool-aid man. However, salience construal – largely realized though gestures – activates force and rhythm as relevant zones for this comparative mapping. In example (7), the rhythmical motion is evoked by the steady train of echolalia in collaboration with a slow, wavy, up-and-down right-hand gesture (shown in Figure 10a). This is contrasted to the already introduced type of echolalia, the Kool-Aid Man, accompanied in example (7) by an explosive forward-moving gesture from the shoulder with an open hand (Figure 10b). As Mittelberg points out “speakers may readily use their heads, manual gestures, or postures to (re)enact force qualities they have experienced or imagine experiencing” (2018, p. 10), and for these gestures, HB uses not only her hand and arm, but also her posture to indicate the force of her echolalia. For the steady train, she is relaxed and leaned back, and for the Kool-Aid Man she leans forward. These examples demonstrate HB’s use of force contours associated with two entities (steady train and the Kool-Aid Man) to evoke comparative construal. The construed force qualities additionally prompt possible social responses associated with these entities. While a train chugs along in a predictable pattern, the Kool-Aid Man bursts through walls in an unexpected (and inappropriate) fashion.

Fig. 10a. that’s like [the steady train of echolalia]

Fig. 10b. that’s [the Kool-Aid Man]

HB also describes selective mutism. Selective mutism is characterized by the “absence of speech in almost all […] or only certain situations” and has been shown to be common in ASD (Steffenburg, Steffenburg, Gillberg, & Billstedt, Reference Steffenburg, Steffenburg, Gillberg and Billstedt2018, p. 1163). Both echolalia and selective mutism thus involve the making (or not making) of sound, often speech. Unlike echolalia, which HB describes as forceful outward motion, selective mutism is portrayed as interrupted motion or lack of motion. In example (8) she uses an ice metaphor.

The gesture that coincides with you just freeze up in example (8) is shown in Figures 11a and 11b. HB holds her hands at her throat and lifts her head slightly, as if trying to get sound out. Jointly with the metaphorical verbal expression, the gesture’s position at the throat is suggestive of speech being stopped in the vocal tract and thus, through zone activation, the gesture reinforces relevant zones for the verbally expressed ice metaphor.

Fig. 11a. you just [freeze up]

Fig. 11b. you just [freeze up]

Through speech and gestures, HB describes echolalia in terms of forceful motion and selective mutism in terms of lack of motion or contained motion. Her body is portrayed as a container with an inside (hidden/invisible/inaudible) and an outside (public/visible/audible). A boundary separating these (e.g., a fourth wall) can be traversed through sounds and movements. Essential to her accounts of echolalia are motion, rhythm, and force.

6. Implications for research on figurative meaning in ASD

In this study, we considered discursively situated and thematically organized verbal, gestural, and pictorial data published with the expressed intention of mediating subjective autism experiences to other people. Our results indicate that figurative meaning in this work is malleable and multifunctional, conveys diverse experiences, and transcends single modes of communication. Metaphorization, for instance, is often realized through language and gestures or pictures jointly, as when HB talks of sounds that you grab while gesturally enacting grabbing, or when she writes You speak butterflies. I hear bats. alongside drawings of these animals. Language, gestures, and pictures contribute different nuances of meaning, so that even when their contributions reinforce the same figurative meaning, these are far from redundant (El Refaie, Reference El Refaie2014, p. 152). Additionally, literally used meanings via one mode of communication can be complemented by figurative meanings via another mode, as when HB uses gestures metonymically to complement literal speech. It is well known from previous studies that distinguishing between literal and figurative language is not a simple matter (e.g., Hartman & Paradis, Reference Hartman and Paradis2018; Paradis, Reference Paradis2004, Reference Paradis, Zenker and Gärdenfors2015; Winter, Reference Winter, Speed, O’Meara, Roque and Majid2019). When we consider multimodal communication, including language (spoken or written), gestures, and pictures, as in our data, these difficulties are compounded by the fact that meanings (literal and figurative) are communicated simultaneously via different modalities as “single integrated expression[s] of meaning” (McNeill, Reference McNeill1992, p. 79, on gesture and speech; El Refaie, Reference El Refaie2019, for verbo-visual metaphor). In our view, a broader focus on figurative meaning in communication can be valuable to research on figurative meaning in ASD. If we study language alone (and comprehension only), we may miss significant means of figurative communication used by individuals with ASD (and also more generally).

To express her experiences, HB draws upon a number of metaphors that are arguably conventionalized in English, such as the conduit metaphor. In her work, these metaphors are instantiated verbally, gesturally, and pictorially. Her use of metaphor, not least in gestural expressions, strongly suggests that she employs such figurative mappings in a manner that is meaningfully embodied. Gestures reinforce mappings by providing “an embodied subjective experience of [a] metaphorical source domain” (Müller, Reference Müller and Hampe2017a, p. 299), and in HB’s work metaphoricity is not only foregrounded, but also perceptually anchored, in the sense that a “gesture is an embodied experience” (2017a, p. 299, italics in original). Conventional metaphors are at times regarded in ASD research as constituting rather stable and fixed form–meaning pairings that can simply be retrieved from memory. The embodied (experientially based) use of conventionalized metaphor in our data suggests that research on figurative meaning in ASD could benefit from viewing conventionality of figurative meaning as “an issue of discourse and interaction, of language in use” (2017a, p. 298), as opposed to a stable feature of figurative mappings. In addition, as El Refaie points out in her discussion of metaphor in graphic novels, conventional mappings can be used in creative ways for “discovery of new connections between two areas of experience and the imaginative reinterpretation of conventional metaphors” (Reference El Refaie2014, p. 152), so there is room for creativity in the use of conventional mappings as well. It is of significance to consider not only the presence or absence of figurative mappings, but also the manner and mode through which they are expressed in communication.

Some characteristics of HB’s figurative meaning expression are apparent only when we take a comprehensive view of her work. Certain concepts, such as anchoring, are employed over and over again to cover different experiences. Extensive use of single concepts for multiple experiences can be seen as limiting compared to using multiple concepts to describe the same experiences, but it can also be seen as highly creative because it optimizes the elasticity of these concepts. In order to capture this type of creative ability, it could be advantageous to complement experimental research on figurative meaning in ASD with qualitative studies of naturally occurring figurative meaning use by individuals with ASD.

Our study also indicates that research in this field could benefit from a focus on not only comprehension of isolated metaphors, but on patterns of figurative meanings in use. In our data, binary opposition (e.g., motionrest, containmentrelease) expressed through language, gestures, and pictures facilitates mediation of a range of dynamic experiences, many of which are tied to control.

ASD constitutes a spectrum and capacity for figurative reasoning and expression likely varies to a great extent between individuals. In addition to successfully mediating her experiences verbally and gesturally, HB skillfully expresses nuances of these experiences pictorially. However, the study of drawing(s) is not limited to investigations of advanced artwork, like HB’s, but could be used more broadly in ASD research on figurative meaning.

7. Conclusion

To investigate the role of figurative meaning in ASD, researchers need to consider both production and comprehension of meaning, ideally through a range of methods and diverse data types. Such an expanded investigative scope should take figurative meaning as its focus as opposed to figurative language and include different modes of communication. In this study, we carefully consider one autistic artist’s use of language, co-speech gestures, and pictures and we couch our observations in a coherent analytical framework. We focus on three prominent themes in the published videos and comics and find that experiential meaning is expressed figuratively through varied and multimodal use of a range of construals of salience and comparison. The use of figurative meaning in our data paints a different picture than much previous research, which has presented primarily deficits in figurative meaning comprehension in ASD. As a case study of meaning production, our study does not contradict research on figurative meaning comprehension in populations, but it indicates benefits of an approach that takes a broader perspective on figurative meaning and that involves consideration of more than one mode of communication.

Supplementary materials

For supplementary materials for this paper, please visit <http://doi.org/10.1017/langcog.2020.20>.

Footnotes

This research was supported by the Kamprad Family Foundation, grant 20190045. We would like to thank the editor and two anonymous reviewers for helpful comments. We would also like to thank HB for giving us permission to include her work in this paper.

[1] Weighted blankets and vests are commonly used as part of occupational therapy interventions in ASD.

[2] HB writes dissociate in the bottom left panel. In most of her work, however, she refers to this form of absorption as lucid dreaming (a type of dissociative experience).

[3] In storytelling, characters break the fourth wall when they address/acknowledge their audience, thereby potentially interfering with the suspension of disbelief.

References

references

American Psychiatric Association (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Arlington, VA: American Psychiatric Publishing.Google Scholar
Bogdashina, O. (2005). Communication issues in autism and Asperger syndrome: Do we speak the same language? London: Jessica Kingsley Publishers.Google Scholar
Chahboun, S., Vulchanov, V., Saldaña, D., Eshuis, H. & Vulchanova, M. (2016). Can you play with fire and not hurt yourself? A comparative study in figurative language comprehension between individuals with and without Autism Spectrum Disorder. PLOS ONE 11(12), e0168571.CrossRefGoogle Scholar
Chahboun, S, Vulchanov, V., Saldaña, D., Eshuis, H. & Vulchanova, M. (2017). Can you tell it by the prime? A study of metaphorical priming in high-functioning autism in comparison to matched controls. International Journal of Language and Communication Disorders 52(6), 766785.CrossRefGoogle Scholar
Cienki, A. (2005). Image schemas and gesture. In Hampe, B. (ed.), From perception to meaning: image schemas in Cognitive Linguistics (pp. 421441). Berlin & New York: Mouton De Gruyter.CrossRefGoogle Scholar
Cienki, A. (2008). Why study metaphor and gesture? In Cienki, A. & Müller, C. (eds), Metaphor and gesture (pp. 525). Amsterdam: John Benjamins.CrossRefGoogle Scholar
Cienki, A. & Müller, C. (2008). Metaphor, gesture, and thought. In Gibbs, R. W. (ed.), The Cambridge handbook of metaphor and thought (pp. 483501). Cambridge: Cambridge University Press.CrossRefGoogle Scholar
Deignan, A., Littlemore, J. & Semino, E. (2013). Figurative language, genre and register. Cambridge: Cambridge University Press.Google Scholar
de Marchena, A. & Eigsti, I.-M. (2010). Conversational gestures in autism spectrum disorders: asynchrony but not decreased frequency. Autism Research 3(6), 311322.CrossRefGoogle Scholar
de Marchena, A., Kim, E. S., Bagdasarov, A., Parish-Morris, J., Maddox, B. B., Brodkin, E. S. & Schultz, R. T. (2019). Atypicalities in gesture form and function in autistic adults. Journal of Autism and Developmental Disorders 49, 14381454.CrossRefGoogle ScholarPubMed
de Ruiter, M. B., Elzinga, B. M. & Phaf, R. H. (2006). Dissociation: cognitive capacity or dysfunction? Journal of Trauma & Dissociation 7(4), 115134.CrossRefGoogle ScholarPubMed
Eigsti, I.-M. (2013). A review of embodiment in autism spectrum disorders. Frontiers in Psychology 4, e00224.CrossRefGoogle ScholarPubMed
Eigsti, I.-M. & de Marchena, A. (2017). Perspectives on gesture from autism spectrum disorder: alterations in timing and function. Behavioral and Brain Sciences 40, e53.CrossRefGoogle ScholarPubMed
Eigsti, I.-M., de Marchena, A., Schuh, J. M. & Kelley, E. (2011). Language acquisition in autism spectrum disorders: a developmental review. Research in Autism Spectrum Disorders 5, 681691.CrossRefGoogle Scholar
El Refaie, E. (2009). Metaphor in political cartoons: exploring audience responses. In Forceville, C. & Urios-Aparisi, E. (eds), Multimodal metaphor (pp. 173196). Berlin: Mouton de Gruyter.Google Scholar
El Refaie, E. (2014). Looking at the dark and bright side: creative metaphors of depression in two graphic memoirs. a/b: Auto/Biography Studies 29(1), 149174.CrossRefGoogle Scholar
El Refaie, E. (2019). Visual metaphor and embodiment in graphic illness narratives. New York: Oxford University Press.CrossRefGoogle Scholar
Forceville, C. (2002). The identification of target and source in pictorial metaphors. Journal of Pragmatics 34(1), 114.CrossRefGoogle Scholar
Forceville, C. (2005). Visual representations of the idealized cognitive model of anger in the Asterix album La Zizanie. Journal of Pragmatics 37, 6988.CrossRefGoogle Scholar
Forceville, C. (2016a). Mixing in pictorial and multimodal metaphors? In Gibbs, R. W. Jr. (ed.), Mixing metaphor (pp. 223239). Amsterdam: Benjamins.Google Scholar
Forceville, C. (2016b). Conceptual metaphor theory, blending theory and other cognitivist perspectives on comics. In Cohn, N. (ed.), The visual narrative reader (pp. 89114). Amsterdam: Benjamins.Google Scholar
Forceville, C. (2017). Book review of Michał Szawerna, Metaphoricity of conventionalized diegetic images in comics: a study in multimodal Cognitive Linguistics. LaMiCuS 2, 275279.Google Scholar
Forceville, C. & Paling, S. (2018). The metaphorical representation of depression in short, wordless animation films. Visual Communication <https://doi.org/10.1177/1470357218797994>.CrossRefGoogle Scholar
Gibbs, R. W. (2019). Metaphor as dynamical–ecological performance. Metaphor and Symbol 34(1), 3344.CrossRefGoogle Scholar
Giora, R., Gazal, O., Goldstein, I., Fein, O. & Stringaris, A. (2012). Salience and context: interpretation of metaphorical and literal language by young adults diagnosed with Asperger’s Syndrome. Metaphor and Symbol 27, 2254.CrossRefGoogle Scholar
Grandin, T. (1995). Thinking in Pictures. London: Bloomsbury.Google Scholar
Hartman, J. (2017) Premonitory urges and Touretting volcanoes: force construal in personal narratives on Tourette Syndrome. Review of Cognitive Linguistics 15(1), 155183.CrossRefGoogle Scholar
Hartman, J. & Paradis, C. (2018) Emotive and sensory simulation through comparative construal. Metaphor and Symbol 33(2), 123143.CrossRefGoogle Scholar
H, B. (2011–2019). autisticinnerspace: thoughts from inside the shell. Retrieved from <//autisticinnerspace.wordpress.com>..>Google Scholar
H, B. (2015–2018). CuriosityRocks: autistic & awesome. Retrieved from <//www.youtube.com/channel/UCzF-2vxXRvpcWGMpVXL4xkw>..>Google Scholar
Hobson, R. P. (2012). Autism, literal language and concrete thinking: some developmental considerations. Metaphor and Symbol 27(1), 421.CrossRefGoogle Scholar
Johnson, M. (1987). The body in the mind: the bodily basis of meaning, imagination and reason. Chicago: University of Chicago Press.CrossRefGoogle Scholar
Kalandadze, T., Norbury, C., Nærland, T. & Næss, K. B. (2018). Figurative language comprehension in individuals with autism spectrum disorder: a meta-analytic review. Autism 22(2), 99117.CrossRefGoogle ScholarPubMed
Kamio, Y. & Toichi, M. (2000). Dual access to semantics in autism: Is pictorial access superior to verbal access? Journal of Child Psychology and Psychiatry 41(7), 859867.CrossRefGoogle ScholarPubMed
Kana, R. K., Keller, T. A., Cherkassky, V. L., Minshew, N. J. & Just, M. A. (2006). Sentence comprehension in autism: thinking in pictures with decreased functional connectivity. Brain 129, 24842493.CrossRefGoogle ScholarPubMed
Kasirer, A. & Mashal, N. (2014). Verbal creativity in autism: comprehension and generation of metaphoric language in high-functioning autism spectrum disorder and typical development. Frontiers in Human Neuroscience 8, e00615.CrossRefGoogle ScholarPubMed
Kennedy, J. M. (1982). Metaphor in pictures. Perception 11, 589605.CrossRefGoogle ScholarPubMed
Kövecses, Z. (2000). Metaphor and emotion: language, culture, and body in human feeling. Cambridge: Cambridge University Press.Google Scholar
Kurtz, L. C., Trainer, S., Beresford, M., Wutich, A. & Brewis, A. (2017). Blogs as elusive ethnographic texts: methodological and ethical challenges in qualitative online research. International Journal of Qualitative Methods 16, 112.CrossRefGoogle Scholar
Lakoff, G. (1987). Women, fire and dangerous things: what categories reveal about the mind. Chicago: University of Chicago Press.CrossRefGoogle Scholar
Lakoff, G. (1990). The invariance hypothesis: Is abstract reasoning based on image schemas? Cognitive Linguistics 1(1), 3974.CrossRefGoogle Scholar
Lakoff, G. (1996). Sorry, I’m not myself today: the metaphor system for conceptualizing the self. In Fauconnier, G. & Sweetser, E. (eds), Spaces, worlds, and grammar (pp. 91123). Chicago: University of Chicago Press.Google Scholar
Lakoff, G. (1999). Philosophy in the flesh: the embodied mind and its challenge to Western thought. New York: Basic Books.Google Scholar
Lakoff, G. & Johnson, M. (1980). Metaphors we live by. Chicago: University of Chicago Press.Google Scholar
Littlemore, J. (2017). On the role of embodied cognition in the understanding and use of metonymy. In Hampe, B. (ed.), Metaphor: embodied cognition and discourse (pp. 160177). Cambridge: Cambridge University Press.CrossRefGoogle Scholar
Littlemore, J. (2019). Metaphors in the mind: sources of variation in embodied metaphor. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
MacKay, G. & Shaw, A. (2004). A comparative study of figurative language in children with autistic spectrum disorders. Child Language Teaching and Therapy 20(1), 1332.CrossRefGoogle Scholar
Mashal, N. & Kasirer, A. (2011). Thinking maps enhance metaphoric competence in children with autism and learning disabilities. Research in Developmental Disabilities 32, 20452054.CrossRefGoogle ScholarPubMed
Mashal, N. & Kasirer, A. (2012). Principal component analysis study of visual and verbal metaphoric comprehension in children with autism and learning disabilities. Research in Developmental Disabilities 33, 274282.CrossRefGoogle ScholarPubMed
McNeill, D. (1992). Hand and mind: what gestures reveal about thought. Chicago: University of Chicago Press.Google Scholar
Melogno, S., D’Ardia, C., Pinto, M. A. & Levi, G. (2012). Metaphor comprehension in autistic spectrum disorders: case studies of two high-functioning children. Child Language Teaching and Therapy 28(2), 177188.CrossRefGoogle Scholar
Mittelberg, I. (2018). Gestures as image schemas and force gestalts: a dynamic systems approach augmented with motion-capture data analyses. Cognitive Semiotics 11(1), e0002.CrossRefGoogle Scholar
Mittelberg, I. & Waugh, L. R. (2009). Metonymy first, metaphor second: a cognitive-semiotic approach to multimodal figures of thought in co-speech gesture. In Forceville, C. & Urios-Aparisi, E. (eds), Multimodal metaphor (pp. 329356). Berlin: Mouton de Gruyter.Google Scholar
Müller, C. (2017a). Waking metaphors: embodied cognition in multimodal discourse. In Hampe, B. (ed.), Metaphor: embodied cognition and discourse (pp. 297316). Cambridge: Cambridge University Press.CrossRefGoogle Scholar
Müller, C. (2017b). How recurrent gestures mean: conventionalized contexts-of-use and embodied motivation. Gesture 16(2), 276303.CrossRefGoogle Scholar
Müller, C. & Cienki, A. (2009). Words, gestures, and beyond: forms of multimodal metaphor in the use of spoken language. In Forceville, C. & Urios-Aparisi, E. (eds), Multimodal metaphor (pp. 297328). Berlin: Mouton de Gruyter.Google Scholar
Olofson, E. L., Casey, D., Oluyedun, O. A., Van Herwegen, J., Becerra, A. & Rundblad, G. (2014). Youth with Autism Spectrum Disorder comprehend lexicalized and novel primary conceptual metaphors. Journal of Autism and Developmental Disorders 44, 25682583.CrossRefGoogle ScholarPubMed
Paradis, C. (2004). Where does metonymy stop? Senses, facets and active zones. Metaphor and Symbol 19(4), 245264.CrossRefGoogle Scholar
Paradis, C. (2005). Ontologies and construals in lexical semantics. Axiomathes 15, 541573.CrossRefGoogle Scholar
Paradis, C. (2011). Metonymization: a key mechanism in semantic change. In Benczes, R., Barcelona, A. & de Mendoza Ibáñez, F. J. Ruiz (eds), Defining metonymy in Cognitive Linguistics: towards a consensus view (pp. 6188). Amsterdam & Philadelphia: John Benjamins.CrossRefGoogle Scholar
Paradis, C. (2015). Conceptual spaces at work in sensuous cognition: domains, dimensions and distances. In Zenker, F. & Gärdenfors, P. (eds), Applications of conceptual spaces: the case of geometric knowledge representation (pp. 3355). Dordrecht: Springer Verlag.Google Scholar
Potsch, E. & Williams, R. F. (2012). Image schemas and conceptual metaphors in action comics. In Bramlett, F. (ed.), Linguistics and the study of comics (pp. 1336). London: Palgrave.CrossRefGoogle Scholar
Pouw, W. J. L., de Nooijer, J. A., van Gog, T., Zwaan, R. A. & Paas, F. (2014). Toward a more embedded/extended perspective on the cognitive function of gestures. Frontiers in Psychology 5, e00359.CrossRefGoogle Scholar
Reddy, M. J. (1979). The conduit metaphor—a case of frame conflict in our language about language. In Ortony, A. (ed.), Metaphor and thought (pp. 284324). Cambridge: Cambridge University Press.Google Scholar
Rundblad, G. (2017). Metaphor acquisition and use in individuals with neurodevelopmental disorders. In Semino, E. & Demjén, Z. (eds), The Routledge handbook of metaphor and language (pp. 486502). New York: Routledge.Google Scholar
Rundblad, G. & Annaz, D. (2010a). The atypical development of metaphor and metonymy comprehension in children with autism. Autism 14(1), 2946.CrossRefGoogle Scholar
Rundblad, G. & Annaz, D. (2010b). Development of metaphor and metonymy comprehension: receptive vocabulary and conceptual knowledge. British Journal of Developmental Psychology 8, 547563.CrossRefGoogle Scholar
Shinohara, K. & Matsunaka, Y. (2009). Pictorial metaphors of emotion in Japanese comics. In Forceville, C. J. & Urios-Aparisi, E. (eds), Multimodal metaphor (pp. 265293). Berlin: Mouton de Gruyter.Google Scholar
Steffenburg, H., Steffenburg, S., Gillberg, C. & Billstedt, E. (2018). Children with autism spectrum disorders and selective mutism. Neuropsychiatric Disease and Treatment 14, 11631169.CrossRefGoogle ScholarPubMed
Stiegler, L. N. (2015). Examining the echolalia literature: Where do speech-language pathologists stand? American Journal of Speech-Language Pathology 24, 750762.CrossRefGoogle ScholarPubMed
Van Herwegen, J. & Rundblad, G. (2018). A cross-sectional and longitudinal study of novel metaphor and metonymy comprehension in children, adolescents, and adults with autism spectrum disorder. Frontiers in Psychology 9, e00945.CrossRefGoogle ScholarPubMed
Vulchanova, M., Saldaña, D., Chahboun, S. & Vulchanov, V. (2015). Figurative language processing in atypical populations: the ASD perspective. Frontiers in Human Neuroscience 9, e00024.CrossRefGoogle ScholarPubMed
Winter, B. (2019). Synaesthetic metaphors are neither synaesthetic nor metaphorical. In Speed, L. J., O’Meara, C., Roque, L. San & Majid, A. (eds), Perception metaphors (pp. 105126). Amsterdam: John Benjamins.CrossRefGoogle Scholar
Zwaan, R. (2014). Embodiment and language comprehension: reframing the discussion. Trends in Cognitive Sciences 18(5), 229234.CrossRefGoogle ScholarPubMed
Figure 0

table 1. Overview of data

Figure 1

table 2. Questions for consideration of figurative meaning in verbal, gestural, and pictorial communication

Figure 2

Fig. 1a. [grabs the words …]

Figure 3

Fig. 1b. … and [anchors] them down for me

Figure 4

Fig. 1c. on the way

Figure 5

Fig. 1d. [to my ear]

Figure 6

Fig. 2a. they’re [slightly too] loud

Figure 7

Fig. 2b. [I’m controlling the noise]

Figure 8

Fig. 3. Comic No. 112: Auditory Processing Disorder

Figure 9

Fig. 4. Comic No. 42: Anchor Points

Figure 10

Fig. 5a. [my vision]

Figure 11

Fig. 5b. [and I feel all floaty]

Figure 12

Fig. 6. Comic No. 120: Little Miss ExistentialisT

Figure 13

Fig. 7. Comic No. 26: Squash

Figure 14

Fig. 8. Comic No. 44: The fantasy zone is my Everyday

Figure 15

Fig. 9a. … sort of [projects it outwards] …

Figure 16

Fig. 9b. … rather than [focus inwards] …

Figure 17

Fig. 9c. … I [project it outwards]

Figure 18

Fig. 10a. that’s like [the steady train of echolalia]

Figure 19

Fig. 10b. that’s [the Kool-Aid Man]

Figure 20

Fig. 11a. you just [freeze up]

Figure 21

Fig. 11b. you just [freeze up]

Supplementary material: File

Hartman and Paradis supplementary material

Hartman and Paradis supplementary material 1

Download Hartman and Paradis supplementary material(File)
File 18.1 KB
Supplementary material: File

Hartman and Paradis supplementary material

Hartman and Paradis supplementary material 2

Download Hartman and Paradis supplementary material(File)
File 13.9 KB