Hostname: page-component-77c89778f8-fv566 Total loading time: 0 Render date: 2024-07-17T12:57:23.689Z Has data issue: false hasContentIssue false

Gaze and eye movement in dialogue interpreting: An eye-tracking study

Published online by Cambridge University Press:  26 May 2020

Elisabet Tiselius*
Affiliation:
Department of Swedish Language and Bilingualism, Institute for Interpreting and Translation Studies, Stockholm University
Kayle Sneed
Affiliation:
Department of Swedish Language and Bilingualism, Institute for Interpreting and Translation Studies, Stockholm University
*
Address for correspondence: Elisabet Tiselius E-mail: elisabet.tiselius@su.se
Rights & Permissions [Opens in a new window]

Abstract

Previous studies have investigated the cognitive processes of simultaneous interpreting and translation using eye-tracking. No study has yet utilized eye-tracking to investigate cognitive load and cognitive effort in dialogue interpreting. An eye-tracking study was conducted on two groups of interpreters (experienced and inexperienced) with varying language backgrounds during a staged dialogue interpreting session. The aim of the study was to explore gaze patterns in dialogue interpreting in relation to the interpreters’ action and translation direction. The results indicated there were differences in gaze patterns depending on the action and the language used. Participants averted gaze more when interpreting into the allophone language (the L2 for a majority of the participants in this study). This may indicate that interpreting into L2 in a dialogue may involve more cognitive effort than interpreting into L1. Finally, gaze patterns did not differ significantly between inexperienced and experienced dialogue interpreters.

Type
Research Article
Creative Commons
Creative Common License - CCCreative Common License - BYCreative Common License - NCCreative Common License - ND
This is an Open Access article, distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives licence (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is unaltered and is properly cited. The written permission of Cambridge University Press must be obtained for commercial re-use or in order to create a derivative work.
Copyright
Copyright © The Author(s), 2020. Published by Cambridge University Press

Introduction

Dialogue interpreting is a type of interpreting used in encounters with few (often no more than two) participants. It can be defined as a type of very short consecutive interpreting, where the interpreter interprets immediately after a finished turn. The dialogue interpreter interprets both participants’ turns at talk throughout the interpreted interaction, working into and out of both of the languages used in the encounter. During interpreting, the interpreter must understand a message produced in one language and then simultaneously (or else subsequently) reproduce that message in the other language. This means that interpreting requires parallel activation of at least two languages (cf. Dong & Lin, Reference Dong and Lin2013; de Groot & Christoffels, Reference De Groot and Christoffels2007; Englund Dimitrova & Hyltenstam, Reference Englund Dimitrova and Hyltenstam2000; Valdés & Angelelli, Reference Valdés and Angelelli2003). In simultaneous interpreting the process takes place while the speaker continues to produce utterances in the source language, while in consecutive interpreting the production starts once the speaker pauses. Because of the multitasking involved, simultaneous interpreting is often regarded as a form of extreme language management that requires an extensive cognitive effort (Babcock & Vallesi, 2017).

Directionality in interpreting refers to which language the interpreter interprets into, their first language (L1) or their second language (L2). Many researchers argue that interpreting into L1 is the only direction that provides high standard interpreting (for a discussion, see Godijns & Hinderdael, Reference Godijns and Hinderdael2005). It has been argued (e.g., by Gile, Reference Gile, Godijns and Hinderdael2005) that interpreting into L2 increases the cognitive load at the expense of output quality.

In contrast to other types of interpreting in which the interpreter produces output in one language for a longer period of time (e.g., simultaneous or long consecutive interpreting), dialogue interpreting requires active switching between languages for short periods of time. This requires that both languages be cognitively activated, both in terms of perception/comprehension and transfer/production, throughout the interpreted interaction. In the dialogue, each of the interpreter's target language renditions serves as the impetus for a new source language segment (that is, a turn at talk produced by an interlocutor). Dialogue interpreters also monitor their own comprehension of the primary parties’ utterances as well as the primary parties’ understanding of the interpreter's target language utterances. The dialogue interpreter must therefore always have two languages activated for both comprehension and production.

Considering the many simultaneous tasks of dialogue interpreting, it is as demanding as simultaneous interpreting in terms of advanced bilingual language processing. Yet, unlike the case with simultaneous interpreting, dialogue interpreting has mostly been studied from interactional or sociological perspectives (Englund Dimitrova & Tiselius, Reference Englund Dimitrova, Tiselius and Muñoz Martín2016). Compared to simultaneous interpreting, the cognitive characteristics of dialogue interpreting are far less known. We argue that dual or multi-tasking in dialogue interpreting is different, but not less demanding, than other types of interpreting.

Dialogue interpreting often occurs in the public service sector, in encounters between an allophone-speaking individual and representatives of the majority language–speaking population.Footnote 1 The population of dialogue interpreters is also often different to the population of simultaneous or consecutive interpreters often studied with a cognitive approach. Dialogue interpreters are often late L2 learners of one of their working languages, without a background in language studies. In many countries they are so-called heritage language speakers, meaning they grew up speaking the allophone language at home (Mellinger & Gasca-Jimenéz, Reference Mellinger and Gasca-Jiménez2019). They are often part of the immigrant population of the country in question. Many dialogue interpreters also have different levels of fluency in their two languages, something that might affect the cognitive load and interpreting process (Englund Dimitrova & Tiselius, Reference Englund Dimitrova, Tiselius and Muñoz Martín2016). These characteristics also make them different from a traditional directionality perspective.

As has been described above, dialogue interpreters differ from simultaneous and consecutive interpreters, especially in terms of directionality. Furthermore, it can be argued that dialogue interpreting is as cognitively demanding as simultaneous or longer consecutive interpreting. We therefore consider that it is necessary to study the dialogue interpreter's cognitive load through the lens of directionality.

Cognitive load and effort in interpreting

In this article, cognitive load is defined according to Sweller's cognitive load theory (Sweller, Ayres & Kalyuga, Reference Sweller, Ayres, Kalyuga, Sweller, Ayres and Kalyuga2011) as the load imposed on the working memory either by information (intrinsic load) or by the manner in which the information is presented (extraneous load). Our definition of cognitive (or mental) effort, on the other hand, draws on Russo and Dosher's (Reference Russo and Dosher1983) definition of cognitive effort as the total amount of cognitive resources needed to complete a task. Following these two definitions, we define cognitive load as the external conditions to which the interpreters’ working memory is exposed and cognitive effort as the amount of resources the interpreter uses in order to carry out the interpreting task.

Just like with any mental process, interpreting requires a certain amount of cognitive effort (Gile, Reference Gile1999). Sometimes during an interpreted event the cognitive load may increase, thus requiring a higher level of cognitive effort. Seeber and Kerzel (Reference Seeber and Kerzel2011) give examples of this on the syntactic level when languages do not correspond syntactically and require an interpreter to apply different strategies (such as stalling production in order to wait for a final verb, or anticipating a word or a meaning-bearing unit) in order to restructure the message. An increased cognitive load during an interpreted dialogue may also be due to directionality, conversation management, monitoring of comprehension, or the various emotional challenges that may be present in any dialogue, be it bilingual or monolingual, interpreted or not (Frith, Reference Frith2012).

Research has demonstrated that gaze patterns may provide insight into cognitive load and cognitive effort. For example, in research into gaze patterns in monolingual conversations Vredeveldt, Hitch and Baddeley (Reference Vredeveldt, Hitch and Baddeley2011) found that recall was significantly better when subjects closed their eyes. They argue that this is evidence that eye closure reduces cognitive load. Doherty-Sneddon and Phelps’ (Reference Doherty-Sneddon and Phelps2005) study involved children and found that the primary function of gaze aversion during question–answer sequences was to manage the cognitive load involved in the processing of environmental information. To our knowledge, gaze aversion has not been studied in interpreting, but these results indicate that gaze aversion or even eye closure are indications of handling cognitive load, as they may be used to concentrate or to trigger memory.

Gaze patterns in interpreting have been investigated with a main focus on conference interpreters (Seeber, Reference Seeber, Zybatov, Petrova and Ustaszewski2012; Stachowiak, Reference Stachowiak2017). Stachowiak (Reference Stachowiak2017) found that eye movements and beat gestures change when language-related cognitive load or congruence between visual non-linguistic and auditory linguistic input changes in interpreting; these findings may also be relevant to dialogue interpreting. Stachowiak concluded that both lists and numbers increase cognitive load, deriving evidence for the former from gestures and evidence for the latter from eye movements. Seeber (Reference Seeber, Zybatov, Petrova and Ustaszewski2012) investigated the relation between fixations on visual information and audio information. He found that interpreters used support from visual information for large numbers.

Dialogue interpreters use gaze in many different ways, for instance to monitor turn-taking or to indicate understanding or misunderstanding (Mason, Reference Mason, Baraldi and Gavioli2012). In dialogue interpreting, mobile eye-trackers have been used to study how gaze affects feedback, so-called backchanneling (Vranjes, Brône & Feyaerts, Reference Vranjes, Brône and Feyaerts2018), while video recordings have been used to study how gaze patterns affect inclusion in the interpreted conversation (Krystallidou, Reference Krystallidou2014) and how turn-taking affects participation (Bot, Reference Bot2005; Davitti, Reference Davitti2013; Mason, Reference Mason, Baraldi and Gavioli2012). None of these studies investigate gaze patterns in relation to cognitive load. To our knowledge, gaze patterns have not previously been studied in dialogue interpreting in relation to directionality of interpreting. Given that gaze patterns shed light on increased cognitive load in monolingual speech, investigating gaze patterns and their relation to cognitive load in dialogue interpreting may be of interest as well. Moreover, previous research on differences between interpreters with and without experience shows that experienced interpreters seem to handle cognitive load better (Liu, Reference Liu, Hansen, Chesterman and Gerzymisch-Arbogast2008). Gaze patterns and interpreters’ handling of cognitive load may develop and change with experience. Therefore, it seems justified to also investigate differences in gaze patterns between experienced and inexperienced dialogue interpreters and what it may tell us about cognitive load.

Directionality in dialogue interpreting

Directionality is an important research area in interpreting studies, mainly in regard to conference interpreters (Godijns & Hinderdael, Reference Godijns and Hinderdael2005). In conference interpreting research and pedagogy, there has been a divide between the view that an interpreter should work only into the L1 and the view an interpreter should work only into the L2 (Dose, Reference Dose2014; Godijns & Hinderdael, Reference Godijns and Hinderdael2005, p. 2). Regardless of the validity of either of these claims, they both assume that the interpreter has a choice – that it is possible for the interpreter to work in one direction but not in the other. As discussed above, bidirectional interpreting is a fundamental characteristic of the dialogue interpreter's task. Also, many (although far from all) dialogue interpreters are late learners of their L2 and may have asymmetrical language proficiency in their two languages (Tiselius & Englund Dimitrova, Reference Tiselius and Englund Dimitrova2019).

Directionality has been investigated both from a process perspective, investigating variations in cognitive load and mental effort related to direction, and from a product perspective, mainly focusing on the quality of the interpreting product in the different directions. Results from process studies in conference interpreting indicate that the cognitive load increases when working into an L2 (e.g., Chang, Reference Chang2009; de Bot, Reference De Bot, Englund Dimitrova and Hyltenstam2000; Hyönä, Tommola & Alaja, Reference Hyönä, Tommola and Alaja1995; Kurz, Reference Kurz, Snell-Hornby, Pöchhacker and Kaindl1994). In her study, Bartłomiejczyk (Reference Bartłomiejczyk2006) found that strategies differed depending on language direction. This was confirmed by Chang and Schallert (Reference Chang and Schallert2007), who note that interpreters who regularly work in both directions develop different strategies for each direction, such as increased omissions and generalizations when working into L2. Chang and Schallert also mention the effects of language asymmetries on strategies. Chmiel (Reference Chmiel2016) found activation in L2 to be equally fast for uni-directional and bi-directional interpreters, while bi-directional interpreters had faster activation in L1.

In terms of product, Van Dijk, Boers, Christoffels and Hermans (Reference Van Dijk, Boers, Christoffels and Hermans2011) found interpreting into spoken Dutch to be of higher quality regardless of the participants’ L1, whether Dutch SL or spoken Dutch. Dose (Reference Dose2014) found that quality increased when the participants were familiar with the topic, regardless of direction. Napier, Rohan and Slater (2005) investigated sign language interpreters’ direction preference and self-perceived L1. Most of their participants identified as late learners of sign language. However, in contrast to the L1 norm in interpreting described above, a majority of respondents preferred working into sign language, their L2. In the case of sign language interpreting, working into L2 is the norm, which may contribute to this result.

In conclusion, research has shown that, for conference interpreting, directionality has an impact on cognitive load, strategy use, and quality. To the best of our knowledge, the effects of directionality and its implication for cognitive load have not been investigated in dialogue interpreting, although there are references in the literature to the possible impact of language asymmetry on dialogue interpreting (Herring, Reference Herring2018).

The present study

The aim of this study is to investigate dialogue interpreters’ gaze during task performance. The interpreters’ gaze patterns in dialogue interpreting presumably follow the same patterns as in any dialogue: that is, shifting between gazing at the speaker, gazing at other participants, or gazing randomly (Argyle & Cook, Reference Argyle and Cook1976; Kendon, Reference Kendon1967). Gaze patterns may also be a tool for turn-taking (Oertel, Wlodarczak, Edlund, Wagner & Gustafsson, Reference Oertel, Wlodarczak, Edlund, Wagner and Gustafson2013; Vertegaal, Slagter, van der Veer & Nijholt, Reference Vertegaal, Slagter, Van der Veer and Nijholt2001). Also, as described above, gaze patterns may indicate a response to increased cognitive load. The difference between gaze patterns in dialogues and interpreted dialogues relate, among other things, to the interpreter's need to handle the interpreted event (Mason, Reference Mason, Baraldi and Gavioli2012).

In an interpreted event, we can assume that cognitive load, and consequently cognitive effort, differs depending on directionality, especially if the competence levels of the two languages are asymmetrical. Furthermore, interpreting experience may also have an impact on the cognitive load, given that interpreters’ management of their cognitive resources presumably becomes more skillful as the interpreter gains experience (Liu, Reference Liu, Hansen, Chesterman and Gerzymisch-Arbogast2008). An experienced interpreter may also have more context-related competence, which may also alleviate cognitive load.

In this study we explore the gaze patterns of experienced and inexperienced interpreters in relation to their actions (listening and producing), as well as language direction, with the aim of investigating the cognitive aspects of gaze patterns during a dialogue. Our research questions were as follows:

  1. 1. What is the interpreter's gaze pattern during dialogue interpreting?

  2. 2. Does the gaze pattern depend on the language direction?

  3. 3. Are there any observable differences in the gaze pattern between experienced and inexperienced interpreters?

This study is exploratory. We hope to shed light both on the cognitive effort inherent in dialogue interpreting and possible impacts of directionality on cognitive load/effort; we also suspect that the interpreter's level of experience may affect cognitive load/effort. Inasmuch as the use or impact of gaze in turn-taking has been investigated in previous studies, we do not examine it in this study.

Methods

Participants

Interpreters were recruited as participants (n = 17), and were classified respectively as either inexperienced (n = 10) or experienced (n = 7) interpreters (see Table 1). All participants had Swedish as one of their working languages, and either French, Polish, or Spanish as the other. The inexperienced interpreters were last-term students or recent graduates in public service interpreting,Footnote 2 and the experienced interpreters were Swedish state-authorized interpreters with at least four years of experience.

Table 1. Descriptive data for interpreter participants

To establish language proficiency, the participants answered background questions about their language background and daily language use. Ten participants reported having Swedish as their strongest language or L1, two reported that their two languages were equally strong (one French-speaking inexperienced participant and one Spanish-speaking experienced participant), and five reported that Swedish was the weaker language (one experienced Spanish-speaking interpreter and the four Polish-speaking participants). Participants also took the Dialang (Diagnostic Language Assessment System for Adult Learners) test. The Dialang test is a test developed to diagnose a candidate's language level following the Council of Europe's framework for language proficiency, CEFER (Huhta, Luoma, Oscarson, Sajavaara, Takala & Teasdale, Reference Huhta, Luoma, Oscarson, Sajavaara, Takala and Teasdale2002), and is also available online.Footnote 3 The Dialang test was chosen for easy access and distribution, and as a complement to self-reports, which some researchers have found not to be entirely accurate (e.g., Onna & Jansen, Reference Onna and Jansen2006; for detailed information about the process of establishing participants’ language proficiency, see Tiselius & Englund Dimitrova, Reference Tiselius and Englund Dimitrova2019). The Dialang test is not available in Polish, so the Polish participants took Dialang for Swedish only. Dialang assessed that all the participants had Swedish language proficiency at the C1–C2 level, thus indicating that all participants’ Swedish language skills were either as strong as or stronger than in their other language.

The interpreting task

The interpreters’ task was to interpret a simulated job-counseling consultation (an excerpt of the role-play can be found in appendix 1). The interpreted event took place in a meeting room at Stockholm University and was simulated by actors, one part being a Swedish employment officer, and the other one playing a newly-arrived, non-Swedish speaker seeking help to find a job and take Swedish classes. It took the participants roughly fifteen minutes to interpret the event. All participants received information about the context of the role-play (i.e., job counselling for a newly arrived immigrant) one week before the recordings. The role-play was prepared with potentially cognitively challenging passages in regard to terminology, enumerations (both numbers and lists), emotions (anger and sadness), disrespect of turn-taking, long turns, face-threats (both to primary parties and the interpreter), speed, and comprehension.

Seven actors were given the prepared manuscript for the role-play: two Swedish-language actors took turns playing the role of the employment officer, while five foreign-language actors took turns playing the role of the job seeker in French, Spanish, and Polish, respectively. The actors were assigned to the role-plays on the basis of availability. Appendix 2 shows the distribution of the different actors over the role-plays. Swedish actor 1 did not know the allophone language in the role-plays, while Swedish actor 2 did (Polish). The non-Swedish actors were all mother-tongue speakers of the allophone language. The French and Polish actors knew Swedish, while the Spanish actors had limited knowledge of Swedish. All actors used the same manuscript, and they were encouraged to act out the role-play and leave the manuscript if the interpreting deviated from it. In such cases they were to follow the interpreters and try to return to the script when possible. This solution was chosen to create comparable data for the different interpreters.

Data collection procedure

The interpreted event was recorded with two video cameras, and the interpreter wore eye-tracking glasses. The participants were placed in the classical equilateral-equiangular triangle seating for dialogue interpreting, with the two primary participants sitting opposite each other and the interpreter at the triangle's third point. The interpreter participants were instructed not to take notes, both in order to ensure comparable data and to not run the risk of losing eye-tracking data when the interpreters were reading back their notes.

The study was performed using SMI Glasses 2.0 with the SMI Smart RecorderS4 at a sampling rate of 60 Hz. In the few eye-tracking studies conducted to explore dialogue interpreting, the focus has primarily been on interaction and natural data (Vranjes, Bot, Feyaerts & Brône, Reference Vranjes, Bot, Feyaerts, Brône, Brône and Oben2018; Vranjes, Brône & Feyaerts, Reference Vranjes, Brône and Feyaerts2018). In translation process research that uses eye-tracking, cognitive effort has been linked to difference in fixation patterns (Jakobsen, Reference Jakobsen, Schwieter and Ferreira2017). Eye-tracking studies in translation use a desk-top eye-tracker, often combined with key-logging data and screen recordings in order to follow the translators’ gaze and keyboard activities during the translation task. This type of data can provide insights about the translation process and cognitive load in written translation. But in interpreting, and particularly in dialogue interpreting, the process differs from written translation in terms of the context and the fact that the process both starts and ends with oral representations in the two languages. In dialogue interpreting, therefore, data collection for investigating cognitive effort and the interpreting process is set up differently. In order to preserve the natural body language and movement associated with dialogue interactions, we used a mobile eye-tracker. Despite the reduced sampling rate of mobile eye-trackers, this model is ideal for dialogue interpretation. It allows the wearer to interact with the world around them in a much more natural and holistic way than is possible with remote or tower-mounted eye-trackers. Each participant was first fitted with the glasses. Then a three-point calibration procedure was performed using triangulated dots on a wall at roughly the same distance as the speakers in the dialogue interpreting session. The calibration was then tested by having each participant look at a specified point in the room (e.g., the corner of a notebook, the thumb of the experimenter's hand). If the fixation dot did not match the assigned point of interest, the calibration procedure was repeated.

Data analysis

Raw eye data from the participants was first processed using SMI BeGaze 3.6 software. Fixations were manually classified into five areas of interest (AOI): client face, Swedish face, client hands, Swedish hands, and not-face. The AOIs were chosen based on common areas of focus during dialogues. As this also coincides with areas where individuals gaze in dialogues, it seemed interesting to pursue these areas more in detail. In order to ensure overall data accuracy and quality, any fixation that appeared outside the eye-tracker's camera view of the scene was discarded as this was likely due to poor eye-tracking accuracy during a period of head movement or peripheral gaze. Given the exploratory nature of the study and the low number of participants, no minimum fixation duration was applied to the data. Additionally, while average fixation duration is a common tool in reading and translation research, our study is not concerned with individual fixations, as on words, but rather the overall gaze pattern of participants to the faces of those they are interpreting for.

Participant actions classified automatically by the eye-tracking software were then confirmed independently by two researchers in BeGaze through annotations (actions), defined as speaking to the client, speaking Swedish, listening to the client, and listening to Swedish. The actions were chosen according to the dialogue interpreters’ typical actions. After processing, the data was exported in a CSV format and analyzed in R 3.4.2.

Time intervals for each fixation were matched up with annotation time intervals (actions) as well as AOIs. Then, the counts of fixations to each AOI for each participant were calculated for each action. In order to simplify the model, and because the current study does not focus on hand movements, fixations to either the client's hands or the Swedish speaker's hands were reclassified to the “not-face” category. Finally, a 2 × 4 repeated measures MANOVA was fitted to the data, with fixations to the client's face, fixations to the Swedish speaker's face, and fixations to neither face constituting the three outcome variables. The given group (experienced/inexperienced) and action (interpreting into Swedish, interpreting to the client, listening to Swedish, listening to the client) comprised the levels of the MANOVA.

Results

Below we present the results of the analysis of the eye tracking data. Table 2 lists the mean number of fixations/gaze to each AOI (allophone client's face, Swedish speaker's face, no face) by action for experienced and inexperienced interpreters. The results of the 2 × 4 repeated measures MANOVA indicated that while action was highly predictive of gaze to AOI (F = 15.270, p < 0.0001), differences related to experience were not significant (F = 2.047, p = 0.117).

Table 2. Fixations/gaze by area of interest (AOI) and action; mean number of fixations with standard deviation given in parenthesis.

In order to understand pairwise differences between actions, 3 repeated measures ANOVAs were conducted without the group variable and with fixations to each AOI as a single outcome variable, and then posthoc pairwise comparisons were made based on the results from the ANOVA measures. The Bonferroni correction method was used to correct for inflated type I error.

There were significantly more gazes to the client's face when both listening to the client (z = -2.731, p = 0.038) and interpreting for the client (z = -3.099, p = 0.012) than when interpreting into Swedish and listening to Swedish (z = -2.814, p = 0.029). All other comparisons on gaze to the client's face were not significant (see Table 3).

Table 3. Comparison of action pairs during fixations on area of interest (client face, Swedish face or no face)

Table 3 also shows that there were significantly more gazes at the Swedish employment officer's face when listening to Swedish than when listening to the allophone language (z = 3.823, p = 0.001) or speaking the allophone language (z = -3.875, p = 0.001). Furthermore, there were also significantly more gazes to the Swedish employment officer's face when listening to Swedish than when speaking Swedish (z = 0.029, p = 0.029). The other comparisons of gaze to the employment's face were not significant.

Finally, gazing at no face (gaze aversion) was significantly more common when speaking the allophone language than all three other actions (listening to allophone: z = 5.857, p < 0.0001; listening to Swedish: z = 3.165, p = 0.009; speaking Swedish: z = -4.477, p < 0.0001). Additionally, gaze aversion was more common when listening to Swedish than when listening to the allophone language (z = 2.693, p = 0.043). No other comparisons on gazing at no face/gaze aversion were significant.

This means that when the interpreters were gazing at the client, they were more likely to be listening to the allophone speaking client or interpreting to the client. When gazing at the Swedish speaker, interpreters were most likely to listen to Swedish. Finally, interpreters were averting gaze (gazing at no face) when they were interpreting into the allophone language.

Differences between the groups (experienced vs. inexperienced) were not statistically present as either a main effect or an interaction: however, numerical differences between groups can be seen in the data. Experienced interpreters seemed to gaze in equal amounts to the allophone speaker's face and avert gaze when listening to the allophone speaker, while inexperienced interpreters seemed to gaze to the allophone speaker's face more. On the other hand, experienced interpreters seemed to avert gaze more when interpreting both into Swedish and into the allophone language than inexperienced interpreters.

Discussion

The aim of this study was to investigate gaze patterns in dialogue interpreting in relation to directionality and experience, with an underlying assumption that cognitive load could impact gaze patterns.

The results indicate that the interpreters’ gaze pattern differed depending on the type of action and language direction, which may be due to differences in cognitive load and required cognitive effort. Gazing to the speaker's face was more common when interpreting into the allophone language, than when interpreting into Swedish. There may be several other reasons than increased cognitive load/effort to why participants gaze more at the allophone speaker than the Swedish speaker during interpreting. It may be due to power relations in the interpreted event, or increased monitoring of the allophone speakers’ understanding, or simply differences due to culture or education.

When participants interpreted into the allophone language, they averted gaze more than when interpreting into Swedish. The fact that a majority of the participants reported Swedish as their stronger language, or L1, may shed some light on the increased cognitive load and required cognitive effort depending on direction, that is, when working into a weaker language. If we assume that interpreting into the weaker language requires more cognitive effort, as has also previously been suggested (e.g., Hyönä et al., Reference Hyönä, Tommola and Alaja1995; Kurz, Reference Kurz, Snell-Hornby, Pöchhacker and Kaindl1994), then the increased cognitive effort during such an activity could yield a gaze pattern like the one observed in this study. This would support earlier findings on directionality in simultaneous interpreting. Another reason for gaze aversion may also be keeping the turn, and there may be other reasons than directionality for the increased cognitive effort.

Our data also indicate differences in the gaze pattern between our experienced and inexperienced participants. These differences are not significant, yet they represent a trend worth studying on a larger set of data. Experienced participants averted their gaze more than inexperienced participants during all actions, especially when interpreting. Inexperienced participants seemed to gaze more at the allophone client's face when listening.

An important limitation of the study is the small number of participants, which affects statistical power and may be a reason for not finding significant difference between experienced and inexperienced interpreters. Also, the group of inexperienced interpreters comprised both last-term students and recent graduates, creating another possible distortion of results. Furthermore, the need for an experimental setting, and conditions such as using a scripted role-play and not allowing interpreters to take notes, may have an impact on the natural gaze pattern. There may also be other reasons than directionality for the increased cognitive load that were not investigated here, such as difficult terminology or long sentences.

Conclusion

This article has reported on an explorative study on interpreters’ gaze patterns during dialogue interpreting. Cognitive processes of interpreting have previously only been studied during simultaneous interpreting. Previous studies on dialogue interpreting have mainly focused on interactional or sociological aspects.

Our study showed that the action gaze aversion stands out when speaking the allophone language. Gaze aversion may, thus, be an indication of increased cognitive load due to directionality. Our data support previous studies that interpreting into L2 may involve more cognitive effort than interpreting into L1. It may be important for users of dialogue interpreting in public service settings, where the interpreter has often learned the country's majority language as an adult, to understand the increased cognitive effort to interpret into the L2, and the necessity to adjust language, chunks, or terminology, accordingly. Our results may also inform interpreter trainers of the need to adapt training to the fact that directionality has an impact on the interpreting.

Future research possibilities include analyzing gaze patterns in relation to other types of actions in dialogue interpreting, such as during linguistic difficulties (e.g., handling terminology or long sentences or monitoring translation), but also in terms of monitoring other types of events such as the other participants’ comprehension and emotions as well as any disrespect of turn-taking. Furthermore, turn-taking in general in interpreting remains to be investigated from a perspective of cognitive effort. Finally, interpreting in different modalities, such as to and from a signed language, can also be added to the analysis of gaze patterns and cognitive effort in interpreting. Future studies also need to include more participants and different language combinations, including non-Indo-European languages.

Acknowledgements

The authors would like to thank the interpreters who participated in this study, SMI (SensoMotoric Instruments) for kindly providing the eye-tracking glasses, and anonymous reviewers and honest friends for invaluable comments on the manuscript. This article is part of the Invisible Process research project funded by the Swedish Research Council (VR 2016-01118).

Appendix 1. Excerpt from the role-play, omitted parts indicated by […]

Employment officer: Hello and welcome to the employment office. We have not met before, but my name is [________] and I'm your employment officer.

Job seeker: Hello, sorry I'm late, my name is [_____________].

EO: We have an interpreter today, and, I don't know [you turn to the interpreter], do you want to introduce yourself? [you seem like you don't think it's very important]

Interpreter: [Introduction].

EO: Thank you! Very well! Do you understand each other?

JS: Yes.

[…]

JS: OK. I see, but my daughter is very talented, and as I said I'm new here, so it takes some time for me to understand how things work here.

EO: [you interrupt the client before she's ready with her sentence and before the interpreter starts] Yes, I see. But we have to start. We will discuss the introduction task and the introduction benefit. I will tell you more about that later. But maybe you would like to say something about yourself.

JS: [You are a bit confused and concerned that you were interrupted] What do you want me to say?

EO: Well, I've gotten the documentation from the Migration Board, but I haven't had the time to read it all yet. So, I thought it was best if you tell me a little bit about yourself, your education, your children, when you came to Sweden, and when you got your residence permit.

JS: Well. OK. I have three children […]

[…]

EO: That's exactly what I was planning to …

JS: [you interrupt the employment officer before he's finished and before you receive an interpretation, you get annoyed and show that you're worried] But I'm worried too. Are the activities full-time? What about the children? I take care of them alone now. I was separated from their father during the flight and we haven't heard anything from him. [you become very sad and speak in a very low voice] It's really hard, I don't know if he's alive or …

[…]

Appendix 2. Distribution of actors between the role-plays

Footnotes

1 In this article we use the term “allophone,” instead of for example “minority language” speaker, for all languages that are not majority or indigenous to Sweden (Swedish, Sami, and Swedish Sign Language). The term is used in Canada for an immigrant whose first language is neither French nor English. We have chosen “allophone,” as it is unmarked in terms of origin or size of the language spoken. The use of “allophone” in this meaning can for instance be found in the works of Leanza (e.g., Reference Leanza2005)

2 The Polish inexperienced participants were recent graduates, whereas the other groups were still students. The Polish inexperienced participants therefore had received some (although very limited) professional experience at the time of data collection. As we were comparing inexperienced and very experienced interpreters, we did not deem this a crucially distorting factor. The age of the inexperienced interpreters may seem elevated, in particular in the Spanish group. The age span reflects the age of Swedish university students, who are older than the European average (Sadurskis, Reference Sadurskis2018). University education in Sweden is free of charge and there is no age limit.

References

Argyle, M., & Cook, M (1976). Gaze and mutual gaze. London: Cambridge University Press.Google Scholar
Bartłomiejczyk, M (2006). Strategies of simultaneous interpreting and directionality. Interpreting 8, 149174. doi:10.1075/intp.8.2.03barCrossRefGoogle Scholar
Bot, H (2005). Dialogue interpreting in mental health. Amsterdam/New York. Rodopi.Google Scholar
Chang, C.-Y. (2009). Testing applicability of eye-tracking and fMRI to translation and interpreting studies: An investigation into directionality. Ph.D. dissertation, Imperial College London, University of London.Google Scholar
Chang, C., & Schallert, D (2007). The impact of directionality on Chinese/English simultaneous interpreting. Interpreting 9, 137176. doi:10.1075/intp.9.2.02chaCrossRefGoogle Scholar
Chmiel, A (2016). Directionality and context effects in word translation tasks performed by conference interpreters. Poznan Studies in Contemporary Linguistics 52, 269295. doi: 10.1515/psicl-2016-0010CrossRefGoogle Scholar
Davitti, E (2013). Dialogue interpreting as intercultural mediation: Interpreters’ use of upgrading moves in parent-teacher meetings. Interpreting 15, 168199. doi:10.1075/intp.15.2.02davCrossRefGoogle Scholar
De Bot, K (2000). Simultaneous interpreting as language production. In Englund Dimitrova, B. & Hyltenstam, K. (eds.), Language processing and simultaneous interpreting: Interdisciplinary perspectives. Amsterdam: John Benjamins, pp. 6588. doi:10.1075/btl.40.06botCrossRefGoogle Scholar
De Groot, A. M., & Christoffels, I. K. (2007). Processes and mechanisms of bilingual control: Insights from monolingual task performance extended to simultaneous interpretation. Journal of Translation Studies 10, 1741.Google Scholar
Doherty-Sneddon, G., & Phelps, F. G (2005). Gaze aversion: A response to cognitive or social difficulty? Memory & Cognition 33, 727733. doi:10.3758/BF03195338CrossRefGoogle ScholarPubMed
Dong, Y., & Lin, J (2013). Parallel processing of the target language during source language comprehension in interpreting. Bilingualism: Language and Cognition 16, 682692. doi:10.1017/S1366728913000102CrossRefGoogle Scholar
Dose, S (2014). Putting directionality into context. Stellenbosch Papers in Linguistics Plus 45, 7188. doi:10.5842/45-0-625CrossRefGoogle Scholar
Englund Dimitrova, B., & Hyltenstam, K. (eds.). (2000). Language processing and simultaneous interpreting: Interdisciplinary perspectives. Amsterdam: John Benjamins. doi:10.1075/btl.40CrossRefGoogle Scholar
Englund Dimitrova, B., & Tiselius, E. (2016). Cognitive aspects of community interpreting: Toward a process model. In Muñoz Martín, R. (ed.), Reembedding translation process research. Amsterdam: John Benjamins, pp. 195214. doi:10.1075/btl.128.10engCrossRefGoogle Scholar
Frith, C. D (2012). The role of metacognition in human social interactions. Philosophical Transactions of the Royal Society B: Biological Sciences 367, 22132223. doi:10.1098/rstb.2012.0123CrossRefGoogle ScholarPubMed
Gile, D (2005). Directionality in conference interpreting: A cognitive view. In Godijns, R. & Hinderdael, M. (eds.), Directionality in interpreting: The “retour” or the “native”?. pp. 926. Ghent: Communication and Cognition.Google Scholar
Gile, D (1999). Testing the Effort Models’ tightrope hypothesis in simultaneous interpreting: A contribution. Hermes: Journal of Linguistics 23,153172. doi:10.7146/hjlcb.v12i23.25553Google Scholar
Godijns, R., & Hinderdael, M. (eds.). (2005). Directionality in interpreting: The “retour” or the “native”? Ghent: Communication and Cognition.Google Scholar
Herring, R. (2018). “I could only think about what I was doing, and that was a lot to think about”: Online self-regulation in dialogue interpreting. Ph.D. dissertation, University of Geneva.Google Scholar
Huhta, A., Luoma, S., Oscarson, M., Sajavaara, K., Takala, S., & Teasdale, A (2002). DIALANG: A diagnostic language assessment system for adult learners. In Council of Europe, Common European framework of reference for languages: Learning, teaching, assessment. Case studies. Strasbourg: Council of Europe Publishing, pp. 130145.Google Scholar
Hyönä, J., Tommola, J., & Alaja, A.-M (1995). Pupil dilation as a measure of processing load in simultaneous interpretation and other language tasks. Quarterly Journal of Experimental Psychology 48, 598612. doi:10.1080/14640749508401407.CrossRefGoogle ScholarPubMed
Jakobsen, A. L (2017). Translation process research. In Schwieter, J. W. & Ferreira, A. (eds.), The handbook of translation and cognition. Hoboken, NJ: Wiley Blackwell, pp. 2149 doi: 10.1002/9781119241485.ch2Google Scholar
Kendon, A (1967). Some functions of gaze direction in social interaction. Acta Psychologica 32, 125. doi: 10.1002/9781119241485.ch2Google Scholar
Krystallidou, D (2014). Gaze and body orientation as an apparatus for patient inclusion into/exclusion from a patient-centred framework of communication. Interpreter and Translator Trainer 8, 399417. doi:10.1080/1750399X.2014.972033.CrossRefGoogle Scholar
Kurz, I. (1994). A look into the “black box”: EEG probability mapping during mental simultaneous interpreting. In Snell-Hornby, M., Pöchhacker, F. & Kaindl, K. (eds.), Translation studies: An interdiscipline. Amsterdam: John Benjamins, pp. 199207. doi: 10.1075/btl.2.25kurCrossRefGoogle Scholar
Liu, M (2008). How do experts interpret? Implications from research in Interpreting Studies and cognitive science. In Hansen, G., Chesterman, A. & Gerzymisch-Arbogast, H. (eds.), Efforts and models in interpreting and translation research: A tribute to Daniel Gile. Amsterdam: John Benjamins, pp. 159178. doi: 10.1075/btl.80.14liuGoogle Scholar
Leanza, Y (2005). Roles of community interpreters in pediatrics as seen by interpreters, physicians and researchers. Interpreting 7, 167192. doi: 10.1075/bct.9.04leaGoogle Scholar
Mason, I (2012). Gaze, positioning and identity in interpreter-mediated dialogues. In Baraldi, C. & Gavioli, L. (eds.), Coordinating participation in dialogue interpreting Amsterdam: John Benjamins, pp. 177199. doi: 10.1075/btl.102.08masCrossRefGoogle Scholar
Mellinger, C. D., & Gasca-Jiménez, L (2019). Challenges and opportunities for heritage language learners in interpreting courses in the U.S. context. Revista Signos: Estudios de Lingüística 52, 950974. doi: 10.4067/S0718-09342019000300950CrossRefGoogle Scholar
Oertel, C., Wlodarczak, M., Edlund, J., Wagner, P., & Gustafson, J (2013). Gaze patterns in turn-taking. In 13th Annual Conference of the International Speech Communication Association 2012 (INTERSPEECH 2012). Red Hook, NY: Curran, pp. 22432246.Google Scholar
Onna, B. T. M. van., & Jansen, C. J. M. (2006). How multilingual are the Dutch really? On proficiency in Dutch, English, French, and German in Dutch organizations. Belgian Journal of English Language and Literatures 4, 169180.Google Scholar
Russo, J. E., & Dosher, B. A (1983). Strategies for multiattribute binary choice. Journal of Experimental Psychology: Learning, Memory, and Cognition 9, 676696. doi: 10.1037/0278-7393.9.4.676Google ScholarPubMed
Sadurskis, A (2018). Higher education in Sweden: 2018 status report. Stockholm: Swedish Higher Education Authority. https://english.uka.se/download/18.7f89790216483fb85588e86/1534509947612/Report-2018-06-26-higher-education-in-Sweden-2018.pdf (retrieved April 1, 2019 ).Google Scholar
Seeber, K. G., & Kerzel, D (2011). Cognitive load in simultaneous interpreting: Model meets data. International Journal of Bilingualism 16, 228242. doi:10.1177/1367006911402982.CrossRefGoogle Scholar
Seeber, K. G (2012). Multimodal input in simultaneous interpreting: An eye-tracking experiment. In Zybatov, L. N., Petrova, A. & Ustaszewski, M. (eds.), Proceedings of the 1st International Conference TRANSLATA, Translation & Interpreting Research: Yesterday–Today–Tomorrow, May 12–14, 2011, Innsbruck. Frankfurt: Peter Lang, pp. 341347.Google Scholar
Stachowiak, K. (2017). Eye movements and gestures as correlates of language processing in consecutive and simultaneous interpreting. Ph.D. dissertation, Adam Mickiewicz University, Poznan.Google Scholar
Sweller, J., Ayres, P., & Kalyuga, S. (2011). Intrinsic and extraneous cognitive load. In Sweller, J., Ayres, P. & Kalyuga, S., Cognitive load theory: Explorations in the learning sciences, instructional systems and performance technologies (vol. 1), pp. 5769. New York: Springer. doi: 10.1007/978-1-4419-8126-4_5CrossRefGoogle Scholar
Tiselius, E., & Englund Dimitrova, B (2019). Asymmetrical language proficiency in the dialogue interpreting process: Methodological issues. Translation, Cognition & Behavior 2, 305322. doi: 10.1075/tcb.00031.tisCrossRefGoogle Scholar
Valdés, G., & Angelelli, C (2003). Interpreters, interpreting, and the study of bilingualism. Annual Review of Applied Linguistics 23, 5878. doi:10.1017/S0267190503000199.CrossRefGoogle Scholar
Van Dijk, R., Boers, E., Christoffels, I., & Hermans, D. (2011). Directionality effects in simultaneous language interpreting: The case of sign language interpreters in the Netherlands. American Annals of the Deaf 156, 4755.CrossRefGoogle ScholarPubMed
Vertegaal, R., Slagter, R., Van der Veer, G., & Nijholt, A. (2001). Eye gaze patterns in conversations: There is more to conversational agents than meets the eyes. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. New York, NY: ACM Press, pp. 301308. doi: 10.1145/365024.365119CrossRefGoogle Scholar
Vranjes, J., Bot, H., Feyaerts, K., & Brône, G (2018). Displaying recipiency in an interpreter-mediated dialogue: An eye-tracking study. In Brône, G. & Oben, B. (eds.), Eye-tracking in interaction: Studies on the role of eye gaze in dialogue. Amsterdam: John Benjamins, pp. 323329.Google Scholar
Vranjes, J., Brône, G., & Feyaerts, K (2018). Dual feedback in interpreter-mediated interactions: On the role of gaze in the production of listener responses. Journal of Pragmatics 134, 1534.CrossRefGoogle Scholar
Vredeveldt, A., Hitch, G. J., & Baddeley, A. D (2011). Eyeclosure helps memory by reducing cognitive load and enhancing visualisation. Memory & Cognition 39, 12531263. doi: 10.3758/s13421-011-0098 -8CrossRefGoogle ScholarPubMed
Figure 0

Table 1. Descriptive data for interpreter participants

Figure 1

Table 2. Fixations/gaze by area of interest (AOI) and action; mean number of fixations with standard deviation given in parenthesis.

Figure 2

Table 3. Comparison of action pairs during fixations on area of interest (client face, Swedish face or no face)