Hostname: page-component-8448b6f56d-jr42d Total loading time: 0 Render date: 2024-04-19T16:19:50.728Z Has data issue: false hasContentIssue false
  • English
  • Français

Types of iconicity and combinatorial strategies distinguish semantic categories in silent gesture across cultures

Published online by Cambridge University Press:  13 September 2019

GERARDO ORTEGA Open the ORCID record for GERARDO ORTEGA [Opens in a new window]  and
ASLI ÖZYÜREK
GERARDO ORTEGA*
Affiliation:
University of Birmingham
ASLI ÖZYÜREK
Affiliation:
Radboud Universiteit Faculteit der Sociale Wetenschappen
*
*Address for correspondence: e-mail: g.ortega@bham.ac.uk
Get access Rights & Permissions [Opens in a new window]

Abstract

In this study we explore whether different types of iconic gestures (i.e., acting, drawing, representing) and their combinations are used systematically to distinguish between different semantic categories in production and comprehension. In Study 1, we elicited silent gestures from Mexican and Dutch participants to represent concepts from three semantic categories: actions, manipulable objects, and non-manipulable objects. Both groups favoured the acting strategy to represent actions and manipulable objects; while non-manipulable objects were represented through the drawing strategy. Actions elicited primarily single gestures whereas objects elicited combinations of different types of iconic gestures as well as pointing. In Study 2, a different group of participants were shown gestures from Study 1 and were asked to guess their meaning. Single-gesture depictions for actions were more accurately guessed than for objects. Objects represented through two-gesture combinations (e.g., acting + drawing) were more accurately guessed than objects represented with a single gesture. We suggest iconicity is exploited to make direct links with a referent, but when it lends itself to ambiguity, individuals resort to combinatorial structures to clarify the intended referent. Iconicity and the need to communicate a clear signal shape the structure of silent gestures and this in turn supports comprehension.

Keywords

silent gestureiconicitylanguage emergencecombinatorial structureemerging sign language
Type
Article
Information
Language and Cognition , Volume 12 , Issue 1: Special Issue on Iconicity , March 2020 , pp. 84 - 113
Copyright
Copyright © UK Cognitive Linguistics Association 2019 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

references

Barsalou, L. W. (1999). Perceptual symbol systems. Behavioral and Brain Sciences 22(4), 577–609.CrossRefGoogle ScholarPubMed
Barsalou, L. W. (2008). Grounding symbolic operations in the brain’s modal systems. In Semin, G. & Smith, E. (eds.), Embodied grounding: social, cognitive, affective and neuroscientific approaches (pp. 9–42). Cambridge: Cambridge University Press.CrossRefGoogle Scholar
Bell, A. (1984). Language style as audience design. Language in Society 13, 240–250.CrossRefGoogle Scholar
Brentari, D., Renzo, A. Di, Keane, J. & Volterra, V. (2015). Cognitive, cultural, and linguistic sources of a handshape distinction expressing agentivity. Topics in Cognitive Science 7(1), 95–123.CrossRefGoogle ScholarPubMed
Christensen, P., Fusaroli, R. & Tylén, K. (2016). Environmental constraints shaping constituent order in emerging communication systems: structural iconicity, interactive alignment and conventionalization. Cognition 146, 67–80.CrossRefGoogle ScholarPubMed
Chu, M. & Kita, S. (2016). Co-thought and co-speech gestures are generated by the same action generation process. Journal of Experimental Psychology: Learning, Memory, and Cognition 42(2), 257–270.Google ScholarPubMed
Clark, H. & Murphy, G. (1982). Audience design in meaning and reference. In Le Ny, J. F. & Kintsch, W. (eds.), Language and comprehension (pp. 287–300). Amsterdam: North-Holland.CrossRefGoogle Scholar
Cook, S. W. & Tanenhaus, M. K. (2009). Embodied communication: speakers’ gestures affect listeners’ actions. Cognition 113(1), 98–104.CrossRefGoogle ScholarPubMed
Dingemanse, M., Blasi, D. E., Lupyan, G., Christiansen, M. H. & Monaghan, P. (2015). Arbitrariness, iconicity and systematicity in language. Trends in Cognitive Sciences 19(10), 603–615.CrossRefGoogle ScholarPubMed
Fay, N., Arbib, M. & Garrod, S. (2013). How to bootstrap a human communication system. Cognitive Science 37(7), 1356–1367.CrossRefGoogle ScholarPubMed
Galantucci, B. & Garrod, S. (2011). Experimental semiotics: a review. Frontiers in Human Neuroscience, 5. doi.org/10.3389/fnhum.2011.00011.CrossRefGoogle ScholarPubMed
Gibson, E., Piantadosi, S. T., Brink, K., Bergen, L., Lim, E. & Saxe, R. (2013). A noisy-channel account of crosslinguistic word-order variation. Psychological Science 24(7), 1079–1088.CrossRefGoogle ScholarPubMed
Goldin-Meadow, S. & Brentari, D. (2017). Gesture, sign and language: the coming of age of sign language and gesture studies. Behavioral and Brain Sciences 40, E46. doi.org/10.1017/S0140525X15001247.CrossRefGoogle ScholarPubMed
Goldin-Meadow, S., McNeill, D. & Singleton, J. (1996). Silence is liberating: removing the handcuffs on grammatical expression in the manual modality. Psychological Review 103(1), 34–55.CrossRefGoogle ScholarPubMed
Goldin-Meadow, S., So, W. C., Özyürek, A. & Mylander, C. (2008). The natural order of events: how speakers of different languages represent events nonverbally. Proceedings of the National Academy of Sciences of the United States of America 105(27), 9163–9168.CrossRefGoogle ScholarPubMed
Hall, M. L., Ahn, Y. D., Mayberry, R. I. & Ferreira, V. S. (2015). Production and comprehension show divergent constituent order preferences: evidence from elicited pantomime. Journal of Memory and Language 81, 16–33.CrossRefGoogle ScholarPubMed
Hall, M. L., Mayberry, R. I. & Ferreira, V. S. (2013). Cognitive constraints on constituent order: evidence from elicited pantomime. Cognition 129(1), 1–17.CrossRefGoogle ScholarPubMed
Hassemer, J. & Winter, B. (2018). Decoding gestural iconicity. Cognitive Science 42(8), 3034–3049.CrossRefGoogle ScholarPubMed
Haviland, J. (2013). The emerging of nouns in a first generation sign language: specification, iconicity, and syntax. Gesture 13(3), 309–353.CrossRefGoogle Scholar
Hockett, C. (1960). The origin of speech. Scientific American 203, 88–111.CrossRefGoogle Scholar
Hostetter, A. B. & Alibali, M. W. (2008). Visible embodiment: gestures as simulated action. Psychonomic Bulletin & Review 15(3), 495–514.CrossRefGoogle ScholarPubMed
Imai, M. & Kita, S. (2014). The sound symbolism bootstrapping hypothesis for language acquisition and language evolution. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences 369(1651). doi.org/10.1098/rstb.2013.0298.CrossRefGoogle ScholarPubMed
Johnston, T. (2001). Nouns and verbs in Australian sign language: An open and shut case? Journal of Deaf Studies and Deaf Education 6(4), 235–257.CrossRefGoogle Scholar
Kendon, A. (2004). Gesture: visible action as utterance. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
Kimmelman, V., Klezovich, A. & Moroz, G. (2018). Iconicity patterns in sign languages. Retrieved from <https://sl-iconicity.shinyapps.io/iconicity_patterns/>.Google Scholar
Kita, S. & Özyürek, A. (2003). What does cross-linguistic variation in semantic coordination of speech and gesture reveal? Evidence for an interface representation of spatial thinking and speaking. Journal of Memory and Language 48(1), 16–32.CrossRefGoogle Scholar
Klima, E. & Bellugi, U. (1979). The signs of language. Cambridge, MA: Harvard University Press.Google Scholar
Mandel, M. A. (1977). Iconic devices in American Sign Language. In Friedman, A. (ed.), On the other hand: new pespectives on American Sign Language (pp. 57–107). New York: Academic Press.Google Scholar
Masson-Carro, I., Goudbeek, M. & Krahmer, E. (2016). Can you handle this? The impact of object affordances on how co-speech gestures are produced. Language, Cognition and Neuroscience 31(3), 430–440.CrossRefGoogle ScholarPubMed
Meir, I., Aronoff, M., Börstell, C., Hwang, S., Ilkbasaran, D., Kastner, I., … Sandler, W. (2017). The origin of grammatical word order: insights from novel communication systems and young sign languages. Cognition 158, 189–207.CrossRefGoogle Scholar
Meir, I., Aronoff, M., Sandler, W. & Padden, C. A. (2010). Sign languages and compounding. In Scalise, S. & Vogel, I. (eds.), Cross-disciplinary issues in compounding (pp. 301–322). Philadelphia, PA: John Benjamins Publishing Company.CrossRefGoogle Scholar
Meir, I., Sandler, W., Padden, C. & Aronoff, M. (2012). Emerging sign languages. In Marschark, M. & Spencer, P. (eds.), The Oxford handbook of deaf studies, language, and education (Vol. 2, pp. 1–22). Oxford: Oxford University Press. doi.org/10.1093/oxfordhb/9780195390032.013.0018Google Scholar
Micklos, A. (2016). Interaction for facilitating conventionalization: negotiating the silent gesture communication of noun–verb pairs. In Roberts, S., Cuskley, C., McCrohon, L., Barcelo-Coblijn, L., Feher, O. & Verhoef, T. (eds.), The evolution of language: Proceedings of the 11th International Conference (EVOLANG11). New Orleans: EVOLANG. Retrieved from <http://evolang.org/neworleans/papers/143.html>.Google Scholar
Misyak, J., Noguchi, T. & Chater, N. (2016). Instantaneous conventions: the emergence of flexible communicative signals. Psychological Science 27(12). doi.org/10.1177/0956797616661199.CrossRefGoogle Scholar
Motamedi, Y., Schouwstra, M., Culbertson, J., Smith, K. & Kirby, S. (2018). Evolving artificial sign languages in the lab: from improvised gesture to systematic sign. doi.org/10.31234/osf.io/be7qy.CrossRefGoogle Scholar
Müller, C. (2013). Gestural modes of representation as techniques of depcition. In Müller, C., Cienki, A., Ladewig, S., McNeill, D. & Bressem, J. (eds.), Body – language – communication: an international handbook on multimodality in human interaction (pp. 1687–1701). Berlin: De Gruyter Mouton.Google Scholar
Müller, C. (2016). From mimesis to meaning: a systematics of gestural mimesis for concrete and abstract referenital gestures. In Zlatev, J., Sonesson, G. & Konderak, P. (eds.), Meaning, mind and communication: explorations in cognitive semiotics. Frankfurt am Main: Peter Lang.Google Scholar
Nyst, V. (2016). The depiction of size and shape in gestures accompanying object descriptions in Anyi (Côte d’Ivoire) and in Dutch (the Netherlands). Gesture 15(2), 156–191.CrossRefGoogle Scholar
Ortega, G., Sümer, B. & Özyürek, A. (2014). Type of iconicity matters: bias for action-based signs in sign language acquisition. In Bello, P., Guarini, M., McShane, M. & Scassellatie, B. (eds.), 36th Annual Conference of the Cognitive Science Society (pp. 1114–1119). Austin, TX: Cognitive Science Society.Google Scholar
Ortega, G., Sümer, B. & Özyürek, A. (2017). Type of iconicity matters in the vocabulary development of signing children. Developmental Psychology 53(1), 89–99.CrossRefGoogle ScholarPubMed
Özçalişkan, Ş., Lucero, C. & Goldin-Meadow, S. (2016). Does language shape silent gesture? Cognition 148, 10–18.CrossRefGoogle ScholarPubMed
Padden, C., Hwang, S.-O., Lepic, R. & Seegers, S. (2015). Tools for language: patterned iconicity in sign language nouns and verbs. Topics in Cognitive Science 7(1), 81–94.CrossRefGoogle ScholarPubMed
Padden, C., Meir, I., Hwang, S.-O., Lepic, R., Seegers, S. & Sampson, T. (2013). Patterned iconicity in sign language lexicons. Gesture 13(3), 287–305.CrossRefGoogle Scholar
Perlman, M. & Lupyan, G. (2018). People can create iconic vocalizations to communicate various meanings to naïve listeners. Scientific Reports 8(1), 2634. doi.org/10.1038/s41598-018-20961-6.CrossRefGoogle ScholarPubMed
Perniss, P., Thompson, R. L. & Vigliocco, G. (2010). Iconicity as a general property of language: evidence from spoken and signed languages. Frontiers in Psychology 1. doi.org/10.3389/fpsyg.2010.00227.CrossRefGoogle ScholarPubMed
Perniss, P. & Vigliocco, G. (2014). The bridge of iconicity: from a world of experience to the experience of language. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences 369(1651). doi.org/10.1098/rstb.2013.0300.CrossRefGoogle Scholar
Pettenati, P., Sekine, K., Congestrì, E. & Volterra, V. (2012). A comparative study on representational gestures in Italian and Japanese children. Journal of Nonverbal Behavior 36(2), 149–164.CrossRefGoogle Scholar
Pettenati, P., Stefanini, S. & Volterra, V. (2010). Motoric characteristics of representational gestures produced by young children in a naming task. Journal of Child Language 37(4), 887–911.CrossRefGoogle Scholar
Ramachandran, S. & Hubbard, E. M. (2001). Synaesthesia: a window to perception, thought and language. Journal of Consciosness Studies 8(12), 3–34.Google Scholar
Roomer, E. K., Hoogerwerf, A. C. & Linn, D. E. (2011). Boston benoem taak 2011. Utrecht.Google Scholar
Sloetjes, H. & Wittenburg, P. (2018). ELAN (version 5.2). Nijmegen: Max Planck Institute for Psycholinguistics. Retrieved from <https://tla.mpi.nl/tools/tla-tools/elan/>..>Google Scholar
Smith, K. & Kirby, S. (2008). Cultural evolution: implications for understanding the human language faculty and its evolution. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences 363(1509). doi.org/10.1098/rstb.2008.0145.CrossRefGoogle ScholarPubMed
Sulik, J. (2018). Cognitive mechanisms for inferring the meaning of novel signals during symbolisation. PLoS ONE 13(1). doi.org/10.1371/journal.pone.0189540.CrossRefGoogle ScholarPubMed
Supalla, T. & Newport, E. L. (1986). How many sits in a chair? The derivations of nouns and verbs in American Sign Language. In Siple, P. (ed.), Understanding language through sign language research (pp. 91–132). New York: Academic Press.Google Scholar
Tkachman, O. & Sandler, W. (2013). The noun–verb distinction in two young sign languages. Gesture 13(3), 253–286.CrossRefGoogle Scholar
Tolar, T. D., Lederberg, A. R., Gokhale, S. & Tomasello, M. (2008). The development of the ability to recognize the meaning of iconic signs. Journal of Deaf Studies and Deaf Education 13(2), 225–240.CrossRefGoogle ScholarPubMed
van Nispen, K., van de Sandt-Koenderman, M., Mol, L. & Krahmer, E. (2014). Pantomime strategies: on regularities in how people translate mental representations into the gesture modality. In Bello, C., Guarini, M., McShane, M. & Scassellatti, B. (eds.), Proceedings of the 36th Annual Conference of the Cognitive Science Society (CogSci 2014) (pp. 3020–3026). Austin, TX: Cognitive Science Society.Google Scholar
van Nispen, K., van de Sandt-Koenderman, W. M. E. & Krahmer, E. (2017). Production and comprehension of pantomimes used to depict objects. Frontiers in Psychology 8. doi.org/10.3389/FPSYG.2017.01095.CrossRefGoogle ScholarPubMed
Verhoef, T., Kirby, S. & de Boer, B. (2016). Iconicity and the emergence of combinatorial structure in language. Cognitive Science 40(8), 1969–1994.CrossRefGoogle ScholarPubMed
Vigliocco, G., Perniss, P. & Vinson, D. (2014). Language as a multimodal phenomenon: implications for language learning, processing and evolution. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences 369(1651). doi.org/10.1098/rstb.2013.0292.CrossRefGoogle ScholarPubMed