Hostname: page-component-77c89778f8-gq7q9 Total loading time: 0 Render date: 2024-07-17T08:27:14.711Z Has data issue: false hasContentIssue false
  • English
  • Français

Analyzing gaze allocation during language planning: a cross-linguistic study on dynamic events1

Published online by Cambridge University Press:  06 June 2014

MONIQUE FLECKEN ,
JOHANNES GERWIEN ,
MARY CARROLL  and
CHRISTIANE VON STUTTERHEIM
MONIQUE FLECKEN*
Affiliation:
Donders Centre for Cognition, Radboud University Nijmegen, the Netherlands, and Institut für Deutsch als Fremdsprachenphilologie, Heidelberg University, Germany
JOHANNES GERWIEN
Affiliation:
Institut für Deutsch als Fremdsprachenphilologie, Heidelberg University, Germany
MARY CARROLL
Affiliation:
Institut für Deutsch als Fremdsprachenphilologie, Heidelberg University, Germany
CHRISTIANE VON STUTTERHEIM
Affiliation:
Institut für Deutsch als Fremdsprachenphilologie, Heidelberg University, Germany
*
†Address for correspondence: Monique Flecken, Donders Centre for Cognition, Radboud University, Montessorilaan 3, 6525 HR Nijmegen, The Netherlands. e-mail: m.flecken@donders.ru.nl
Get access Rights & Permissions [Opens in a new window]

Abstract

Studies on gaze allocation during sentence production have recently begun to implement cross-linguistic analyses in the investigation of visual and linguistic processing. The underlying assumption is that the aspects of a scene that attract attention prior to articulation are, in part, linked to the specific linguistic system and means used for expression. The present study concerns naturalistic, dynamic scenes (video clips) showing causative events (agent acting on an object) and exploits grammatical differences in the domain of verbal aspect, and the way in which the status of an event (a specific vs. habitual instance of an event) is encoded in English and German. Fixations in agent and action areas of interest were timelocked to utterance onset, and we focused on the pre-articulatory time span to shed light on sentence planning processes, involving message generation and scene conceptualization. Findings are threefold: (i) English speakers mark the status of an event as specific in relation to the action, with progressive aspect marking on the verb in each utterance. German speakers do so by elaborating specific characteristics of the agent; (ii) participants display significantly different gaze allocation patterns to agent and action regions although the sentences produced in both languages follow the same subject−verb word order; and (iii) the analysis of gaze patterns during sentence production given dynamic scenes provide complementary results from a more naturalistic paradigm, to those obtained in studies with still images.

Keywords

language productioncross-linguistic analysiseye-trackingdynamic stimulievent construalgrammatical aspect
Type
Research Article
Information
Language and Cognition , Volume 7 , Issue 1 , March 2015 , pp. 138 - 166
Copyright
Copyright © UK Cognitive Linguistics Association 2014 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

references

Anderson, S., Matlock, T., & Spivey, M. (2013). Grammatical aspect and temporal distance in motion descriptions. Frontiers in Psychology, 4, 1−9.Google Scholar
Baayen, H. (2008). Analyzing linguistic data. Cambrige: Cambridge University Press.Google Scholar
Barr, D. (2012). Walkthrough of an ‘empirical logit’ analysis in R, online: <http://talklab.psy.gla.ac.uk/tvw/elogit-wt.html>..>Google Scholar
Bates, D. (2010). Correlated random effects in lmer and false convergence, online: <https://stat.ethz.ch/pipermail/r-sig-mixed-models/2010q2/003921.html> (last accessed 8 January 2014).+(last+accessed+8+January+2014).>Google Scholar
Bates, D., Maechler, M., & Bolker, B. (2012). Package ‘lme4’. Linear mixed effects models using S4 classes, online: <http://cran.r-project.org/web/packages/lme4/index.html>..>Google Scholar
Beek, v. G., Flecken, M., & Starren, M. (2013). Aspectual perspective-taking in L1 and L2 Dutch. International Review of Applied Linguistics, 51 (2), 199−227.Google Scholar
Bergen, B., & Wheeler, K. (2010). Grammatical aspect and mental simulation. Brain and Language, 112, 150−158.Google Scholar
Berman, R., & Slobin, D. (1994). Relating events in narrative: a crosslinguistic developmental study. Hillsdale, NJ: Erlbaum.Google Scholar
Bock, K., Irwin, D., & Davidson, D. (2004). Putting first things first. In Henderson, J. & Ferreira, F. (Eds.), The interface of language, vision, and action: eye movements and the visual world (pp. 224−250). New York: Psychology Press.Google Scholar
Bock, K., Irwin, D., Davidson, D., & Levelt, W. (2003). Minding the clock. Journal of Memory and Language, 48, 653−685.Google Scholar
Bock, K., & Levelt, W. (1994). Language production: grammatical encoding. In Gernsbacher, M. A. (Ed.), Handbook of psycholinguistics (pp. 945−984). London: Academic Press.Google Scholar
Brown-Schmidt, S., & Konopka, A. (2008). Little houses and casas pequenas: message formulation and syntactic form in unscripted speech with speakers of English and Spanish. Cognition, 109 (2), 274−280.Google Scholar
Brown-Schmidt, S., & Tanenhaus, M. (2006). Watching the eyes when talking about size: an investigation of message formulation and utterance planning. Journal of Memory and Language, 54, 592−609.Google Scholar
Carroll, M., & Stutterheim, C. v. (2011). Event representation, time event relations, and clause structure: a cross-linguistic study of English and German. In Pederson, E. & Bohnemeyer, J. (Eds.), Event representation (pp. 68−83). Cambridge: Cambridge University Press.Google Scholar
Carroll, M., Stutterheim, C. v., & Nüse, R. (2004). The language and thought debate: a psycholinguistic approach. In Habel, C. & Pechmann, T. (Eds.), Approaches to language production (pp. 183−218). Berlin: Mouton de Gruyter.Google Scholar
Coco, M., & Keller, F. (2010). Sentence production in naturalistic scenes with referential ambiguity. In Ohlsson, S. & Catrambone, R. (Eds.), Proceedings of the 32nd Annual Conference of the Cognitive Science Society (pp. 1070−1075). Austin, TX: Cognitive Science Society.Google Scholar
Comrie, B. (1976). Aspect: an introduction to the study of verbal aspect and related problems. Cambridge: Cambridge University Press.Google Scholar
Dahl, Ö. (1995). The marking of the episodic/generic distinction in tense-aspect systems. In Carlson, G. & Pelletier, F. (Eds.), The generic book (pp. 412−425). Chicago: University of Chicago Press.Google Scholar
Dobel, C., Glanemann, R., Kreysa, H., Zwitserlood, P., & Eisenbeiss, S. (2010). Visual encoding of coherent and non-coherent scenes. In Bohnemeyer, J. & Pederson, E. (Eds.), Event representation in language and cognition (pp. 189−215). Cambridge: Cambridge University Press.Google Scholar
Flecken, M. (2011). Event conceptualization by early bilinguals: insights from linguistic and eye tracking data. Bilingualism: Language and Cognition, 14 (1), 61−77.Google Scholar
Flecken, M., & Gerwien, J. (2013). Grammatical aspect modulates event duration estimations: findings from Dutch. In Knauff, M.Pauen, M.Sebanz, N., & Wachsmuth, I. (Eds.), Proceedings of the 35th Annual Conference of the Cognitive Science Society (pp. 2309−2314). Austin, TX: Cognitive Science Society.Google Scholar
Gleitman, L., January, D., Nappa, R., & Trueswell, J. (2007). On the give and take between event apprehension and utterance formulation. Journal of Memory and Language, 57 (4), 544−569.Google Scholar
Griffin, Z. (2004). Why look? Reasons for eye movements related to language production. In Henderson, J. & Ferreira, F. (Eds.), The integration of language, vision, and action: eye movements and the visual world (pp. 213−247). New York: Taylor & Francis.Google Scholar
Griffin, Z., & Bock, K. (2000). What the eyes say about speaking. Psychological Science, 11, 274−279.Google Scholar
Griffin, Z., & Spieler, D. (2006). Observing the what and when of language production for different age groups by monitoring speakers’ eye movements. Brain and Language, 99, 272−288.Google Scholar
v. Heusinger, K. (2002). Specificity and definiteness in sentence and discourse structure. Journal of Semantics, 19 (3), 245−274.Google Scholar
Huettig, F., Rommers, J., & Meyer, A. (2011). Using the visual world paradigm to study language processing: a review and critical evaluation. Acta Psychologica, 137, 151−171.Google Scholar
Jaeger, F., Furth, K., & Hilliard, C. (2012). Incremental phonological encoding during unscripted sentence production. Frontiers in Psychology, 3, 1−22.Google Scholar
Jaeger, T., & Norcliffe, E. (2009). The cross-linguistic study of sentence production: state of the art and a call for action. Language and Linguistics Compass, 3 (4), 866−887.Google Scholar
Klein, W. (1994). Time in language. London: Routledge.Google Scholar
Kuchinsky, S., Bock, K., & Irwin, D. (2011). Reversing the hands of time: changing the mapping from seeing to saying. Journal of Experimental Psychology: Learning, Memory, and Cognition, 37 (3), 748−756.Google Scholar
Levelt, W. (1989). Speaking: from intention to articulation. Cambridge, MA: MIT Press.Google Scholar
Lucy, J. (1992). Grammatical categories and cognition: a case study of the linguistic relativity hypothesis. Cambridge: Cambridge University Press.Google Scholar
Matlock, T. (2010). Abstract motion is no longer abstract. Language and Cognition, 2, 243−260.Google Scholar
Meulen, F. van der, Meyer, A., & Levelt, W. (2001). Eye movements during the production of nouns and pronouns. Memory & Cognition, 29, 512−521.Google Scholar
Meyer, A. (2004). The use of eye tracking in studies of sentence generation. In Henderson, J. & Ferreira, F. (Eds.), The interface of language, vision, and action: eye movements and the visual world (pp. 191−211). New York: Psychology Press.Google Scholar
Meyer, A., & Dobel, C. (2003). Application of eye tracking in speech production research. In Hyöna, J.Radach, J., & Deubel, H. (Eds.), The mind’s eye: cognitive and applied aspects of eye movement research (pp. 253−272). Oxford: Elsevier Science.Google Scholar
Meyer, A., Sleiderink, A., & Levelt, W. (1998). Viewing and naming objects: eye movements during noun phrase production. Cognition, 66, 25−33.Google Scholar
Papafragou, A., Hulbert, J., & Trueswell, J. (2008). Does language guide event perception? Evidence from eye movements. Cognition, 108 (1), 155−184.Google Scholar
Rijkhoff, J., & Seibt, J. (2005). Mood, definiteness and specificity: a linguistic and a philosophical account of their familiarities and differences. Tidsskrift for Sprogforskning, 3 (2), 85−132.Google Scholar
Sauppe, S., Norcliffe, E., Konopka, A., Van Valin, R. Jr., & Levinson, S. (2013). Dependencies first: eye tracking evidence from sentence production in Tagalog. In Knauff, M.Pauen, M.Sebanz, N., & Wachsmuth, I. (Eds.), Proceedings of the 35th Annual Meeting of the Cognitive Science Society (pp. 1265−1270). Austin, TX: Cognitive Science Society.Google Scholar
Slobin, D. (1996). From ‘Thought and language’ to ‘Thinking for speaking’. In Gumperz, J. & Levinson, S. (Eds.), Rethinking linguistic relativity (pp. 70−96). Cambridge: Cambridge University Press.Google Scholar
Soroli, E., & Hickmann, M. (2010). Language and spatial representations in French and in English: evidence from eye-movements. In Marotta, G.Lenci, A.Meini, L., & Rovai, F. (Eds.), Space in language (pp. 581−597). Pisa: Editrice Testi Scientifici.Google Scholar
Strömqvist, S., & Verhoeven, L. (2004), Relating events in narrative. Vol. 2: typological and contextual perspectives. Mahwah, NJ: Erlbaum.Google Scholar
Stutterheim, C. v., Andermann, M., Carroll, M., Flecken, M., & Schmiedtová, B. (2012). How grammaticized concepts shape event conceptualization: insights from linguistic analysis, eye tracking data and memory performance. Linguistics, 50 (4), 833−867.Google Scholar
Stutterheim, C. v, Carroll, M., & Klein, W. (2009). New perspectives in analyzing aspectual distinctions across languages. In Klein, W. & Li, P. (Eds.), The expression of time (pp. 195−216). Berlin: Mouton de Gruyter.Google Scholar
Stutterheim, C. v., & Nüse, R. (2003). Processes of conceptualization in language production: language-specific perspectives and event construal. Linguistics, 41 (5), 831−881.Google Scholar
Talmy, L. (1985). Lexicalization patterns: semantic structure in lexical forms. In Shopen, T. (Ed.), Language typology and syntactic description. Vol. 3: grammatical categories and the lexicon (pp. 57−149). Cambridge: Cambridge University Press.Google Scholar
Tenny, C., & Pustejovsky, J. (2000). A history of events in linguistic theory. In Tenny, C. & Pustejovsky, J. (Eds.), Events as grammatical objects (pp. 3−37). Chicago: University of Chicago Press.Google Scholar