Skip to main content Accessibility help
×
Home
Hostname: page-component-7ccbd9845f-vmftn Total loading time: 1.143 Render date: 2023-01-31T05:09:29.570Z Has data issue: true Feature Flags: { "useRatesEcommerce": false } hasContentIssue true

The attention atlas virtual reality platform maps three-dimensional (3D) attention in unilateral spatial neglect patients: a protocol

Published online by Cambridge University Press:  30 May 2022

Michael Francis Norwood*
Affiliation:
The Hopkins Centre, Menzies Health Institute Queensland, Griffith University, Meadowbrook, QLD, Australia
David Ross Painter
Affiliation:
The Hopkins Centre, Menzies Health Institute Queensland, Griffith University, Meadowbrook, QLD, Australia
Chelsea Hannah Marsh
Affiliation:
The Hopkins Centre, Menzies Health Institute Queensland, Griffith University, Meadowbrook, QLD, Australia School of Applied Psychology, Griffith University, Gold Coast, QLD, Australia
Connor Reid
Affiliation:
Technical Partners Health (TPH), Griffith University, Nathan, QLD, Australia
Trevor Hine
Affiliation:
School of Applied Psychology, Griffith University, Mt Gravatt, QLD, Australia
Daniel S. Harvie
Affiliation:
The Hopkins Centre, Menzies Health Institute Queensland, Griffith University, Meadowbrook, QLD, Australia Innovation, Implementation and Clinical Translation in Health (IIMPACT in Health), Allied Health and Human Performance, University of South Australia, Adelaide, SA, Australia
Susan Jones
Affiliation:
Neurosciences Rehabilitation Unit, Gold Coast University Hospital, Gold Coast, QLD, Australia
Kelly Dungey
Affiliation:
Neurosciences Rehabilitation Unit, Gold Coast University Hospital, Gold Coast, QLD, Australia
Ben Chen
Affiliation:
Allied Health and Rehabilitation, Emergency and Specialty Services, Gold Coast Health, Gold Coast, QLD, Australia
Marilia Libera
Affiliation:
Psychology Department, Logan Hospital, Logan, QLD, Australia
Leslie Gan
Affiliation:
Rehabilitation Unit, Logan Hospital, Meadowbrook, QLD, Australia
Julie Bernhardt
Affiliation:
Florey Institute of Neuroscience and Mental Health, Heidelberg, VIC, Australia
Elizabeth Kendall
Affiliation:
The Hopkins Centre, Menzies Health Institute Queensland, Griffith University, Meadowbrook, QLD, Australia
Heidi Zeeman
Affiliation:
The Hopkins Centre, Menzies Health Institute Queensland, Griffith University, Meadowbrook, QLD, Australia
*
Corresponding author. Email: m.norwood@griffith.edu.au
Rights & Permissions[Opens in a new window]

Abstract

Background:

Deficits in visuospatial attention, known as neglect, are common following brain injury, but underdiagnosed and poorly treated, resulting in long-term cognitive disability. In clinical settings, neglect is often assessed using simple pen-and-paper tests. While convenient, these cannot characterise the full spectrum of neglect. This protocol reports a research programme that compares traditional neglect assessments with a novel virtual reality attention assessment platform: The Attention Atlas (AA).

Methods/design:

The AA was codesigned by researchers and clinicians to meet the clinical need for improved neglect assessment. The AA uses a visual search paradigm to map the attended space in three dimensions and seeks to identify the optimal parameters that best distinguish neglect from non-neglect, and the spectrum of neglect, by providing near-time feedback to clinicians on system-level behavioural performance. A series of experiments will address procedural, scientific, patient, and clinical feasibility domains.

Results:

Analyses focuses on descriptive measures of reaction time, accuracy data for target localisation, and histogram-based raycast attentional mapping analysis; which measures the individual’s orientation in space, and inter- and intra-individual variation of visuospatial attention. We will compare neglect and control data using parametric between-subjects analyses. We present example individual-level results produced in near-time during visual search.

Conclusions:

The development and validation of the AA is part of a new generation of translational neuroscience that exploits the latest advances in technology and brain science, including technology repurposed from the consumer gaming market. This approach to rehabilitation has the potential for highly accurate, highly engaging, personalised care.

Type
Research Protocol
Creative Commons
Creative Common License - CCCreative Common License - BYCreative Common License - NCCreative Common License - ND
This is an Open Access article, distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives licence (https://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is unaltered and is properly cited. The written permission of Cambridge University Press must be obtained for commercial re-use or in order to create a derivative work.
Copyright
© The Author(s), 2022. Published by Cambridge University Press on behalf of Australasian Society for the Study of Brain Impairment

Neglect results in long-term cognitive disability

Unilateral spatial neglect is a lateralised attention disorder characterised by the failure to orient to, attend to, respond to, or report stimuli appearing on the contralesional hemispace (Buxbaum et al., Reference Buxbaum, Ferraro, Veramonti, Farne, Whyte, Ladavas and Coslett2004; Vuilleumier, Reference Vuilleumier2013), despite having intact sensory abilities (Howard & Rowe, Reference Howard and Rowe2018). It is commonplace, long-lasting, but highly variable following stroke (Kaufmann, Cazzoli, Muri, Nef, & Nyffeler, Reference Kaufmann, Cazzoli, Muri, Nef and Nyffeler2020a; Kaufmann et al., Reference Kaufmann, Cazzoli, Pflugshaupt, Bohlhalter, Vanbellingen, Muri and Nyffeler2020b; Harvey, Learmonth, Rossit, & Chen, Reference Harvey, Learmonth, Rossit and Chen2021; Ringman, Saver, Woolson, Clarke, & Adams, Reference Ringman, Saver, Woolson, Clarke and Adams2004) and results in long-term major disability (Checketts et al., Reference Checketts, Mancuso, Fordell, Chen, Hreha, Eskes and Bowen2020; Conti & Amone, Reference Conti and Arnone2016; Wee & Hopman, Reference Wee and Hopman2008). Lateralised frontoparietal neuroanatomical networks, especially in the right hemisphere, are strongly implicated in the pathology (He et al., Reference He, Snyder, Vincent, Epstein, Shulman and Corbetta2007; Pedrazzini & Ptak, Reference Pedrazzini and Ptak2020; Wu et al., Reference Wu, Wang, Zhang, Zheng, Zhang, Rong and Jiang2016). The effectiveness of current treatment approaches is uncertain (Longley et al., Reference Longley, Hazelton, Heal, Pollock, Woodward-Nutt, Mitchell and Bowen2021; Tavaszi, Nagy, Szabo, & Fazekas, 2021; Umeonwuka, Roos, & Ntsiea, Reference Umeonwuka, Roos and Ntsiea2020).

Classical clinical methods assess neglect incompletely

In clinical settings, neglect is typically assessed using simple pen-and-paper tests such as cancellation tasks (Gauthier, Dehaut, & Joanette, Reference Gauthier, Dehaut and Joanette1989; Halligan & Marshall, Reference Halligan and Marshall1989), clock drawing (Freedman, Leach, Kaplan, Shulman, & Delis, Reference Freedman, Leach, Kaplan, Shulman and Delis1994), and line bisection (Albert, Reference Albert1973). However, pen-and-paper tests do not ensure correct identification of moderate neglect, even when several tests are used (Buxbaum et al., Reference Buxbaum, Ferraro, Veramonti, Farne, Whyte, Ladavas and Coslett2004; Buxbaum, Dawson, & Linsley, Reference Buxbaum, Dawson and Linsley2012; Harvey et al., Reference Harvey, Learmonth, Rossit and Chen2021). Consequently, several researchers have recommended computerised approaches that might detect neglect when classical methods cannot (Bonato, Priftis, Umilta, & Zorzi, Reference Bonato, Priftis, Umilta and Zorzi2013; Buxbaum et al., Reference Buxbaum, Dawson and Linsley2012; Ogourtsova, Souza Silva, Archambault, & Lamontagne, Reference Ogourtsova, Souza Silva, Archambault and Lamontagne2017), including virtual reality (VR) (Coyle, Traynor, & Solowij, Reference Coyle, Traynor and Solowij2015; Fordell, Bodin, Bucht, & Malm, Reference Fordell, Bodin, Bucht and Malm2011; Pedroli, Serino, Cipresso, Pallavicini, & Riva, Reference Pedroli, Serino, Cipresso, Pallavicini and Riva2015).

Virtual reality (VR) is useful for cognitive rehabilitation

Simulated and immersive technologies, such as VR, are often ill defined and confused (Gorman & Gustafsson, Reference Gorman and Gustafsson2020). Broadly speaking, VR replaces the perception of reality with a computer stimulation. Modern VR is most commonly a computer-generated, visual simulation of 3D environments, that can be interacted with naturalistically, in real-time, using a headset and hand controllers. The headset occludes perception of the external world, and thus the user experiences a surrounding 3D virtual space, a quality of VR termed immersive. The key risk of VR is motion sickness, but feelings of motion sickness appear to be low in people with stroke (Laver et al., Reference Laver, Lange, George, Deutsch, Saposnik and Crotty2017).

VR is well suited to clinical settings. Within VR we can build complex environments that allow patients to engage in activities that might be impossible or unsafe for them in the real world (Farrow & Reid, Reference Farrow and Reid2004; Kim et al., Reference Kim, Kim, Kim, Chang, Park, Ohn and Kim2007, Reference Kim, Ku, Chang, Park, Lim, Han and Kim2010). These activities can be delivered and monitored by clinicians remotely via telehealth (Burdea, Reference Burdea2003; Morse, Biggart, Pomeroy, & Rossit, Reference Morse, Biggart, Pomeroy and Rossit2020; Threapleton, Drummond, & Standen, Reference Threapleton, Drummond and Standen2016). VR is readily gamified, therefore highly engaging for patients (Pietrzak, Pullman, & McGuire, Reference Pietrzak, Pullman and McGuire2014; Thornton et al., Reference Thornton, Marshall, McComas, Finestone, McCormick and Sveistrup2005) which may facilitate longer rehabilitation sessions, greater adherence to treatment, and better outcomes (Adlakha, Chhabra, & Shukla, Reference Adlakha, Chhabra and Shukla2020; Lohse, Hilderman, Cheung, Tatla, & Van Der Loos, Reference Lohse, Hilderman, Cheung, Tatla and Van Der Loos2014; Parker, Lord, & Needham, Reference Parker, Lord and Needham2013; Huygelier, Mattheus, Vanden Abeele, Van Ee, & Gillebert, Reference Huygelier, Mattheus, Vanden Abeele, Van Ee and Gillebert2021).

VR can map spatial attention and attention problems in neglect

VR can measure spatial attention and map spatial neglect (Buxbaum et al., Reference Buxbaum, Dawson and Linsley2012; Dvorkin, Bogey, Harvey, & Patton, Reference Dvorkin, Bogey, Harvey and Patton2012; Harada & Ohyama, Reference Harada and Ohyama2019; Knobel et al., Reference Knobel, Kaufmann, Gerber, Cazzoli, Muri, Nyffeler and Nef2020). Knobel et al. (Reference Knobel, Kaufmann, Gerber, Cazzoli, Muri, Nyffeler and Nef2020) evaluated the feasibility of a simple visual search task for neglect assessment. The VR game had players search for targets (20 white spheres) located among distractors (100 white cubes). Players were to find all spheres, as quickly as possible, by touching them with the handheld controller, changing their colour to red. Cubes were to be avoided. The players stopped when they stated they had found all spheres. Participants reported the VR as usable with minimal adverse effects. Compared with controls, those with neglect identified targets on the right far more than on the left and were slower overall. There was no significant difference in total right-side targets found between the neglect and control groups. The sensitivity of the VR game and pen-and-paper tests to detect neglect were statistically equivalent. Knobel et al. (Reference Knobel, Kaufmann, Gerber, Cazzoli, Muri, Nyffeler and Nef2020) identify that this study only brought attention to peri-personal (reaching) space, and that employing available technology such as eye-tracking would provide further insight as to patients’ attentional map. For an extended background on neglect, traditional assessments, and VR in neglect see Appendix I.

Introducing the attention atlas (AA)

Overview

It is from this background that we present a new VR platform for neglect assessment, the AA. The AA aims to provide accurate and detailed neglect diagnostics that are accessible to clinicians and patients, evolve our understanding of cognitive impairment following brain injury, and lead to new rehabilitation opportunities.

We plan to critically evaluate the AA in comparison to pen-and-paper methods, for accurate neglect detection and categorisation. Griffith University researchers and clinicians at Gold Coast University Hospital and Logan Hospital (Queensland, Australia) codesigned the VR game within The Hopkins Centre’s Brain and Enriched Environment (BEEhive) Laboratory for cognitive rehabilitation. The AA arose directly from clinical need for new and effective treatments of neglect. The AA represents an in-progress evolution informed by interdisciplinary discussions, extensive playtesting, patient testers within a pre-existing Gold Coast University Hospital Neurosciences Rehabilitation Unit Recreational Activity Program, and detailed clinical feedback at Logan Hospital.

Features and innovations

The AA presents participants with an immersive 3D virtual environment displaying a target amongst several distractors. Building on previous VR neglect assessments, the AA aims to create detailed maps of visuospatial attention and inattention that are precise, accurate, valid, and reliable (Ogourtsova et al., Reference Ogourtsova, Souza Silva, Archambault and Lamontagne2017). The AA adopts recommendations to use eye-tracking (Dvorkin et al., Reference Dvorkin, Bogey, Harvey and Patton2012; Knobel et al., Reference Knobel, Kaufmann, Gerber, Cazzoli, Muri, Nyffeler and Nef2020), quantifies attention in near and far space (Knobel et al., Reference Knobel, Kaufmann, Gerber, Cazzoli, Muri, Nyffeler and Nef2020), employs collaborative design (Morse et al., Reference Morse, Biggart, Pomeroy and Rossit2020), and aims to establish a reference database based on larger sample sizes (Dvorkin et al., Reference Dvorkin, Bogey, Harvey and Patton2012; Knobel et al., Reference Knobel, Kaufmann, Gerber, Cazzoli, Muri, Nyffeler and Nef2020). Finally, psychometric properties (external validity and reliability) of the instrument will be investigated as part of the research programme through detailed quantitative analysis, directly addressing a major recommendation identified from previous VR attempts to assess and treat unilateral spatial neglect (Ogourtsova et al., Reference Ogourtsova, Souza Silva, Archambault and Lamontagne2017). There are several innovations that may maximise the AA’s sensitivity, practicality, and overall effectiveness. Features and innovations include:

  1. 1. Basing the software on established visual search localisation paradigms

  2. 2. Mapping attention using raycasts for continuous and efficient assessment

  3. 3. Implementing eye-tracking (for HTC Vive Pro Eye)

  4. 4. Using a variety of coordinates, search modes, and stimulus parameters

  5. 5. Calibrating an origin to standardise results across players

  6. 6. Incorporating game design principles, including level variety and progression

  7. 7. Allowing games to be based on time limits as well as trial counts

  8. 8. Analysing performance in near-time

  9. 9. Saving game data robustly

  10. 10. Opening software access to facilitate neurorehabilitation research

Visual search paradigm

The AA is depicted in Fig. 1. The player is seated in the real world and perceives a virtual space using a VR headset and hand controller. The AA uses the established cognitive psychology visual search paradigm of locating a single target among distractors (Wolfe & Horowitz, Reference Wolfe and Horowitz2004). At the beginning of each trial, the target is cued centrally, serving to remind the participant of the target and to recentre attention centrally, allowing the identification of potential visuospatial attentional biases, for instance, along the horizontal axis. The search array then appears, and the player is required to locate the target among distractors [see Fig. 1(a)]. The paradigm allows the player to move their head, eyes, and hand to find and point toward the target. In this example, the target is the letter 'T' located among distractor letter 'Ls', a task requiring spatial attention to be allocated serially to each element in turn until the target is found, requiring goal-directed selective attention, which is affected in neglect [see Fig. 1(b)].

Figure 1. The attention atlas. Note. This diagram depicts various search arrays with varying parameters and an icosphere raycast surface positioned with a radius of 1.5 m with respect to the origin (headset position). (a) Trial structure. Cues and arrays are presented until they are selected by the user via pointing and a button press. (b) Visual search paradigm and (c) raycast surface and sources viewed from the third person perspective. The raycast surface is an icosphere (recursion [i.e., subdivision] level = 3) mesh defined by triangular faces and vertices located at the intersections of edges. During the game, the raycast surface is invisible. (d) Raycast surface viewed from a variety of angles. (e) Coordinate systems for array element positioning. (f) Depth configurations. (g) Search modes based on colour variation. (h) Preconfigured stimulus options. Arrays in panels b, e–f have an element inclusion angle of 50.0°. Arrays in panels g–h have an element inclusion angle of 12.5°. Panels e–h are viewed from the first-person perspective. All materials are publicly available: the X bot character model depicted is from Mixamo (https://www.mixamo.com/#/?page=3&type=Character); letters and numbers are from the OpenDyslexic typeface (https://opendyslexic.org/); Georgian symbols are from i2symbol (https://www.i2symbol.com/abc-123/georgian); shapes and balloons are custom-made; playing cards are from Google (https://code.google.com/archive/p/vector-playing-cards/).

Attention mapping using Raycasts

In addition to traditional behavioural measures of RT (ms) and accuracy (% correct) for target localisation, which can be computed by target position, the AA uses raycasts. Raycasts have the advantage of being sampled continuously at the display refresh rate of the VR device (e.g., 90 Hz), documenting the search process implicitly and allowing more efficient attentional mapping than possible using traditional measures, which are acquired more slowly (e.g., at <1 Hz, depending on RTs). Raycasts are ray-to-surface collision tests that can quantify the user’s orientation in 3D space. Rays are cast (i.e., projected) in a straight line from each of the raycast sources (headset, controller, eye gaze) to collide with the raycast surface, a low polygon iscosphere, which surrounds the player [see Fig. 1(c)]. Each raycast source has a ray transform consisting of its origin position 3D (x, y, z) vector and forward direction 3D vector. The raw raycast output is the 3D (x, y, z) hit position on the raycast surface, which is converted into spherical coordinates for analysis. The raycast surface is centred at the origin (headset position), determined via a calibration procedure, described below. When required, raycasts are capable of measuring attention in 360° surrounding the player [see Fig. 1(d)].

AA parameters

The AA incorporates a variety of parameters including coordinate systems, depth configurations, search modes, and stimuli. These parameters will be described in turn. The two coordinate systems position search array elements systematically and at a common radius with respect to the origin (headset position). The spherical coordinate system is based on latitude and longitude and produces horizontally and vertically symmetrical positions. The icosphere coordinate system positions search elements at the icosphere’s vertex positions, which are approximately equivalently spaced in 360°, allowing attentional assessment in front, behind, to the left and right, and above and below the player. Icosphere recursion (i.e., subdivision) level, can vary the density of the positions, and the element inclusion angle can adjust the sampled visual field extent from a central field of view [see Fig. 1(e)].

The AA is configured to compare attention at different depths, with two configurations that use a symmetrical placement of elements in near and far depths and counterbalance polar longitude and latitude coordinates with depth radius. This allows attention (raycasts and traditional measures) to be compared for the same latitude and longitude coordinates at different depths with minimal occlusion from foreground elements. Stereoscopic vision inherently provides the depth cue; maintaining the element’s size across depths can provide an additional optional depth cue [see Fig. 1(f)].

To assess different characteristics of visuospatial attention, the colours of the elements can be varied. For example, elements all presented in a common colour (e.g., white) assess serial spatial attention (Wolfe & Horowitz, Reference Wolfe and Horowitz2004). Presenting the target in a unique colour among homogenously coloured distractors assesses bottom-up attention. Presenting half of the elements in a target colour and the other half in the distractor colour, a conjunction colour-shape task, assesses feature-based attentional filtering of the distractor colour feature (Painter, Dux, Travis, & Mattingley, Reference Painter, Dux, Travis and Mattingley2014). Heterogeneously coloured distractors may increase search difficulty due to task-irrelevant featural variation (e.g., Wei, Yu, Müller, Pollmann, & Zhou, Reference Wei, Yu, Müller, Pollmann and Zhou2019) ['rainbow' mode; see Fig. 1(h)].

Various stimulus options are preconfigured that assess serial search to varying degrees: letters with a target 'T' and distractor 'Ls', as previously described, a target '6' among rotated distractor '6s', a target 'ψ' symbol among distractor Georgian characters, a target balloon without a string among distractor balloons with strings, and a target queen of diamonds playing card among royal cards (jacks, queens, & kings) from all suits (diamonds, hearts, spades, & clubs). Each of these stimuli, when presented uniformly in a white colour, require serial spatial attention to varying extents.

Gameplay loop

As the AA is intended for attention quantification in brain injury populations, the game is made to be as accessible as possible, requiring only that the player point to and select the target. As described, each trial consists of a cue followed by the search array, which is randomly generated. Both the cue and target within the array are selected using the hand controller, which acts as a virtual laser pointer that extends from the controller to the raycast surface, indicated as a yellow beam in-game. The laser pointer highlights that an element is selected by colouring that element yellow. The player presses a button on front of the controller to enter their selection for both the cue and array displays. Feedback rewards correct target localisation via a pleasant sound played through the VR headphones and colourful confetti that appears at the target location. After each response, a new cue appears. The instructions for players including element selection are depicted in Fig. 2.

Figure 2. Instructions for players and element selection. Note. The player indicates that they have found the target by pointing using the controller and pressing the thumb button.

Origin calibration

This origin calibration quantifies the player’s headset position and orientation (i.e., heading direction) within the physical space of the real world, which allows placing the search array and raycast surfaces at common positions relative to the player, irrespective of the headset’s position and orientation. The player (and corresponding virtual VR camera rig) is located near the 3D virtual playing space origin (x = 0, y = 0, z = 0). Four origin targets (spheres, radius = 5 m) are presented with corresponding superimposed arrows at each of the four poles (north, south, east, and west), 100 m from the playing space origin. The player’s task is to move their head (and thus headset) to orient directly toward and raycast collide with one of the origin targets, with their chair in physical space facing toward one of the poles. The use of four poles allows the player to choose a convenient orientation within the physical space [see Fig. 3(a)].

Figure 3. Origin calibration. Note. (a) Origin targets and VR camera rig shown from a top-down perspective. (b) Origin target raycast depicted from the first-person VR perspective.

The player views an origin target sphere (grey), and a superimposed arrow indicates which direction the player should move their head to face the origin target directly. If the angle between the headset raycast transform and origin target position 3D vector is small, the arrow reduces in size, indicating that player is within the intended calibration accuracy (<29.6°) and that they should press a button on the VR controller. The sphere changes colour to white, indicating that the player pressed the button. This triggers a 1 s period where the player’s 3D headset position is recorded as a mean (white sphere). Then, the sphere translates in elevation (y) and position corresponding to latitudinal perspective (x or z, depending on orientation) to match the headset position. The sphere then turns yellow, indicating that the headset raycast collides with the origin target. The player is required to hold this position for 4 s, during which time their headset position is recorded. The mean value is taken as the player’s origin, and stimuli are presented with respect to this location [see Fig. 3(b)]. Origin calibration can be performed once at the start of the game or repeated at the start of each level (i.e., trial block) to account for drift in seated position and to provide rests between levels.

Game design: level variety and progression

Early AA configurations seek to identify the optimal parameters that best distinguish gross neglect from non-neglect. We propose that efficient scanning of the stimulus parameter space will be made possible with raycasts due to their continuous and high sampling rate. Efficiency also allows us to incorporate principles of game design (Bavelier & Green, Reference Bavelier and Green2019; Deterding, Dixon, Khaled, & Nacke, Reference Deterding, Dixon, Khaled and Nacke2011; Shah, Basteris, & Amirabdollahian, Reference Shah, Basteris and Amirabdollahian2014), including variety and progression, which we implement as a series of levels with differing stimulus parameters. Consider an example game, depicted in Fig. 4. This example is based on the spherical coordinate system of 24 element positions arranged in four rings and eight radial arms. Positions that fall outside a central field of view, thus requiring the largest head movements for element identification, are depicted in dark grey [see Fig. 4(b)]. Using an element angle inclusion parameter, it is possible to select and present only a subset of positions on any given trial.

Figure 4. Game and level progression. Note. (a) This example game is based on the spherical coordinate system. Black scatter points reflect array element positions, with a minimal and even spacing of 12.5° on latitude and longitude axes. (b) Example level progression. The game starts with a tutorial and then progresses through a series of four levels, with later levels tending to be more difficult than those proceeding. Each level is comprised of one or more conditions that can be presented in a random order.

Each game is comprised of a series of levels, which can start simple and progressively become more challenging. In contrast to traditional attention studies, which use a set number of trials (e.g., Wolfe, Reference Wolfe1998), each level can last for a fixed duration, which ensures that the AA incorporates players of all abilities within limited clinical schedules. The stimulus parameters within each level can involve randomisation of search conditions across trials, minimising the impact of practice effects on within-level performance comparisons.

The example game starts with a brief tutorial that familiarises users to the search task, with the target defined by a unique feature, making localisation simple. To assess attention along both horizontal and vertical axes, Level 2 intermingles trials with elements presented individually on each of these axes. To assess what stimuli might be most sensitive to neglect, Level 2 contrasts attention to balloons, cards, and symbols. To assess whether neglect affects attention in depth, Level 3 contrasts attention to elements positioned at the same polar coordinates at near (2 m) and far depths (4 m). To map the visual space more completely, Level 4 uses a larger number of target elements. In this example, the duration of each of the levels is proportional to the number of elements and conditions presented [see Fig. 4(b)].

Near-time attention maps and behavioural performance

The AA is designed to provide instantaneous feedback to clinicians on the player’s attentional state so that the AA might be ultimately adapted and personalised for each individual within a gameplay session. Note we use the term near-time as opposed to real-time to indicate that the attention maps are created immediately after rather than during each level. Figure 5 shows an example output for one AA level. The results are collated at the game’s end to provide a single .pdf document of all the level results in the sequence in which they were undertaken. The results show detailed behavioural performance, system performance, and individual-level significance tests for spatial symmetry of attentional raycasts.

Figure 5. Near-time attention maps and behavioural performance. Note. (a) Game start time. (b) Level descriptor. Each level is plotted with all conditions combined (shown here) and for each condition separately (not shown). (c) Raycast scatterplot of raw (x, y, z) data converted to spherical coordinates. The results are presented from the player’s forward-facing perspective. Red reflects headset raycasts, blue reflects controller raycasts, and black reflects eye gaze raycasts. (d) Target detection performance (accuracy and RT). (e) System performance. (f–h) Trial counts and overall behavioural performance as a function of target position. (f) Trials count by target position map. Whiter shades reflect higher counts. (g) Accuracy by target position map. Whiter shades reflect higher accuracy. (h) RT by target position map. Whiter shades reflect faster RTs. (i-k) Raycast heatmaps (2D histograms) for each attentional source. Brighter colours reflect more frequently attended locations. Horizontal and vertical bin size is set to 1°. (i) Raycast headset heatmap. (j) Raycast controller heatmap. (k) Raycast gaze heatmap. (l–n) Latitudinal attentional distribution and symmetry tests for the raycast sources (headset: l, controller: m, gaze: n). Raycast positions are represented as kernel density functions (upper subplots), with inlaid boxplots reflecting the median, interquartile range (IQR), and 1.5 × IQR (whiskers). The significance test for latitudinal symmetry (i.e., a mean central attention not significantly different from zero; lower subplots), is a one sample permutation test, with the sign of latitudinal raycasts randomised on each of N = 100 permutations (α = 0.05, two-tailed). The blue kernel density function reflects the permutation distribution, and the red line indicates the mean obtained value. Significance values are shown in the subplot title, with p < 0.05 indicating significant asymmetries. (o–q) Longitudinal attentional distribution and symmetry tests for the raycast sources (headset: o, controller: p, gaze: q). The procedure is identical to the latitude tests, and the orientation of the plots has been switched to depict vertical attention. (r) Level options. The stimulus configuration that produced the results is reported for evaluation and reference.

Each game is associated with a start time that acts as an anonymous timestamp and game identifier [see Fig. 4(a)]. A level descriptor shows the level name and conditions within each level. Each level is comprised of a series of stimulus options that are presented in multiple repetitions and in random order. This minimises practice effects across levels and facilitates between-condition comparisons. Each level is plotted with all conditions combined and with each condition separately [see Fig. 4(b)]. Raw raycast hit positions are converted into spherical coordinates with a common field of view (rotated appropriately for each calibration orientation) to facilitate comparisons across individuals [see Fig. 4(c)]. Target localisation performance (accuracy and RT), is presented as a function of ordinal trial position to identify potential effects of fatigue [see Fig. 4(d)]. System performance is assessed by analysing the frame rate, which should be close to the maximum for the headset (e.g., 90 Hz). Higher frame rates indicate less input lag, a better player experience, and more temporally detailed raycast data [see Fig. 4(e)]. RTs, accuracy, and number of trials are mapped by target position to identify potential attentional biases, for example on the horizontal axis [see Fig. 4(fh)]. Raycast heatmaps are plotted for each attentional source: headset, control, and gaze, indicating frequently attended locations of 3D space [see Fig. 4(ik)]. Latitudinal attentional distribution and symmetry tests for the raycast sources are conducted to identify potential attentional biases on these axes.

Software architecture

The software is comprised of The Game and The Analyser, two integrated systems that work together in near-time. The Game is programmed in C# using custom-written namespaces, classes, methods, and extensions for The Unity Game Engine (2019.4.20f1), which creates the interactable visual search environments, presents the visual stimuli, and saves the data in a variety of formats including human-readable, cross-software compatible, and C# native binary formats. The Analyser assesses the behavioural performance and raycast data to quantify attention during visual search and provide immediate feedback to clinicians. The Analyser uses Python 3.9.5 [MSC v.1928 64-bit (AMD64)]. The key components of The Game and The Analyser are outlined in Tables 1 and 2, respectively.

Table 1. The Key Game Classes

Table 2. The Key Analyser Modules

Plugins and modules

The SteamVR plugin is used to present the VR and to access the raycast source transforms. Other plugins include the TobiiXR plugin, which allows eye-tracking with the HTC Vive Pro Eye and NumSharp, which allows saving C# arrays directly into Python numpy format. Extenal python modules include pandas, pyarrow, matplotlib, seaborn, astropy, PIL, fpdf, and keyboard. The AA has been tested with the HTC Vive, HTC Vive Pro, HTC Vive Pro Eye, and Oculus Rift CV1.

Save game system

The data is saved in gameplay recordings that are anonymised by date-time. Data are saved with the machine name to allow tracing back to the acquisition site. Each level is separately saved and grouped within a common game folder. The data and results folder and file structure is depicted in Fig. 6.

Figure 6. Data and results folder and file structure. Note: (a) Game data structure. (b) Level data structure. (c) Game results structure. (d) Level results structure.

Code availability

The version of the AA software described in this paper is and will be publicly availability on GitHub (https://github.com/davidrosspainter/TheAttentionAtlas). Additionally, the software will be made publicly available on Open Science Framework (https://osf.io/pa96f/) prior to publication. The software will be made available under the CC BY-SA license, which allows reusers to distribute, remix, adapt, and build upon the material in any medium or format, so long as attribution is given to the creator. The license allows for commercial use and reuse under the same terms.

Research programme

Co-design, feasibility study with users in real-world settings

Our feasibility study employs six focus areas incorporating exploratory mixed-methods analyses in a collaborative design (Bowen et al., Reference Bowen, Kreuter, Spring, Cofta-Woerpel, Linnan, Weiner and Fernandez2009). Mixed-methods has been recommended for feasibility studies of VR use in people’s homes (Laver et al., Reference Laver, Adey-Wakeling, Crotty, Lannin, George and Sherrington2020) and the same applies for hospitals. Morse et al. (Reference Morse, Biggart, Pomeroy and Rossit2020) identifies a need for more mixed-methods studies on patient and clinician perceptions on VR use for neglect, as this approach can enhance our understanding on how to increase therapy engagement.

When creating a health tool for clinical use, Morse et al. (Reference Morse, Biggart, Pomeroy and Rossit2020) recommend involving clinicians, patients, and carers as stakeholders in a collaborative design process; limited engagement with clinicians and patients has been identified in the current literature (Morse et al., Reference Morse, Biggart, Pomeroy and Rossit2020). This guarantees the experiences and concerns of clinicians and patients are included (Santana et al., Reference Santana, Zelinsky, Ahmed, Doktorchik, James, Wilton and Butalia2020) and ensures tools are personalised to the population of interest (Lange, Flynn, Proffitt, Chang, & Rizzo, Reference Lange, Flynn, Proffitt, Chang and Rizzo2010); VR needs to be specifically designed for the stroke population so that various physical and cognitive impairments can be accommodated (Huygelier et al., Reference Huygelier, Mattheus, Vanden Abeele, Van Ee and Gillebert2021).

A useability study should be the first step when testing new VR programmes (Morse et al., Reference Morse, Biggart, Pomeroy and Rossit2020), but to our knowledge only a few studies have measured useability of immersive VR with a neglect population (Knobel et al., Reference Knobel, Kaufmann, Gerber, Cazzoli, Muri, Nyffeler and Nef2020; Morse et al., Reference Morse, Biggart, Pomeroy and Rossit2020; Ogourtsova, Archambault, & Lamontagne, Reference Ogourtsova, Archambault and Lamontagne2019) finding high levels of useability. However, these studies were small scale, and the authors recommend further testing of useability, which was restricted to patients experience only. We aim to test useability on a larger number of patients and with clinicians onsite in a hospital. This allows for ecologically valid testing of the AA. We aim to develop a VR application that is usable and useful for medical and allied health staff and enjoyable and engaging for patients The scale of the current study also allows for ongoing updates to the software based on patient and clinician feedback, in a form of quasi-action-research, before undergoing full testing with people with neglect and people with no neglect. Ultimately, the AA can be further developed into an immersive game experience with maximum usability and clinical function, thereby advancing the field of translational neuroscience.

Hypothesis

We hypothesise that system-level behavioural performance on the AA will produce distinct functional attention maps across one or more spatial dimensions (horizontal, vertical, and depth) for the neurotypical individual and those with brain injury, including for people with neglect.

Design

Studies will address procedural, scientific, and clinical feasibility domains. Each domain is important and distinct where findings from one domain are not required for another. We have incorporated a participatory approach to allow for direct involvement of end users (patients and clinicians) in game design to inform the real-world application of the AA as well as look/feel of the visual search task by incorporating iterative feedback cycles.

The project will be conducted across six separate studies relating to each of the focus areas and three overall aims (see Table 3). Some studies will run in parallel. Findings from this feasibility study will inform the likelihood of successful implementation (and cost-planning) and focus areas for future full-scale efficacy trials and validation studies.

Table 3. Aims and Studies

Aims and studies

Aim 1

Aim one is to ascertain the procedural feasibility in a small-scale demonstration study. Tests with healthy subjects identify a leftward visual field bias, known as pseudoneglect (Friedrich, Hunter, & Elias, Reference Friedrich, Hunter and Elias2018; Jewell & McCourt, Reference Jewell and McCourt2000; Ribolsi, Di Lorenzo, Lisi, Niolu, & Siracusano, Reference Ribolsi, Di Lorenzo, Lisi, Niolu and Siracusano2015). For this aim, we will establish the normal variation (visual field variation, task performance variation) within two 25-min test sessions. A convenience sample of healthy participants aged between 18 and 65 years will trial the AA and provide before and after feedback on the task allowing us to quantify normal variation and measure baseline visualspatial attention within the virtual space. Based on power analysis we will recruit a sample size of 27 (t-test: M = 0, α = 0.05, β = 0.80). Quantitative data will be analysed using descriptive methods (SD, mean, median, percentages), interindividual comparisons of visual field regions of interest and histogram-based ray cast attentional map analysis (Blascheck et al., Reference Blascheck, Kurzhals, Raschke, Burch, Weiskopf and Ertl2017).

Study one will determine acceptability of the AA regarding the experience of motion sickness [using the 12 item Simulator Sickness Questionnaire (Kennedy, Lane, Berbaum, & Lilienthal, Reference Kennedy, Lane, Berbaum and Lilienthal1993)] administered before and after the session, usability [measured using the System Usability Scale (Brooke, Reference Brooke1996; Morse et al., Reference Morse, Biggart, Pomeroy and Rossit2020)] and gaming experience [using the Game Experience Questionnaire (Poels, de Kort, & IJsselsteijn, Reference Poels, de Kort and IJsselsteijn2007)]. Short qualitative interviews will further explore user experience.

Study two will quantify inter- and intra-individual standard variation for visuospatial attention. For all participants, we will measure raycasts, RT, and accuracy. The outcomes of these studies will ascertain the overall functionality of the AA for healthy participants and define the AA parameters that inform sampling and error calculation for future validation studies. Also, we will identify the scope of useful field of view for successful, accurate target searches.

Aim 2

Aim two is to determine the scientific feasibility in a small-scale consecutive case series study. Functionally, we know that neglect varies between individuals (Dvorkin et al., Reference Dvorkin, Bogey, Harvey and Patton2012). Although some patients will present with marked spatial inattention of their visual world, others will present with more subtle but still problematic inattention deficits that remain undetected. This feasibility study will include a consecutive case series of N = 50 people with stroke (inclusive of left and right lesions) identified within 4 weeks of their inpatient rehabilitation admission at the Gold Coast University Hospital Neurosciences Rehabilitation Unit over a consecutive 6-month intake period. A full medical file review and documentation of cognitive, sensory, and motor deficits, noting functional symptoms and any description of neglect and/or visual field difficulties will allow the establishment of a clinical test database to archive symptom recording and clinical performance data.

Study three will first establish the procedural feasibility and acceptability of the AA for clinical samples, applying duplicate methods as Study 1. We aim to determine if participants with brain injury can orientate and perform the instructed tasks (accounting for level of hand mobility, visual problems etc.), and their perceptions of acceptability.

Study four will examine the scientific efficacy of the AA by conducting a small-scale experiment with participants who have a diagnosis of stroke with (>1) and without definite neglect as noted on medical file (retrospective cases with internal controls). For all participants, we will conduct standard neglect assessments (line bisection, letter cancellation, clock drawing). Using the AA, we will measure visuospatial attention for each participant, RT, eye gaze and accuracy, and raycast heatmaps in addition to actual time to complete and any procedural variations noted. Quantitative performance data will be mapped and compared across participants (and compared to standard tests). To determine the capacity to identify potential neglect phenotypes and subgroups from demonstration data if the sample allows, we will apply unsupervised K-means cluster analysis. (Henry, Dymnicki, Mohatt, Allen, & Kelly, Reference Henry, Dymnicki, Mohatt, Allen and Kelly2015) This exploratory data analysis will provide valuable insights for further hypothesis generation. Aim two will allow the creation of preliminary neglect performance datasets and accompanying patterns of symptoms for a consecutive sample. For each participant, we will create the first neglect maps by converting raycasts from Cartesian to spherical coordinates.

Aim 3

Aim three will establish the clinical feasibility in a small-scale observational study and qualitative value study. For a rehabilitation tool to be of value to rehabilitation services, patients and clinicians must trust its practical application and potential to guide clinical management (Zeeman, Reference Zeeman, Zeeman, Kendall and Wright2013). A separate study will be conducted with hospital inpatients to establish clinical feasibility of the platform.

Study five will conduct a preliminary, small-scale case-control observational study to establish clinical implementation feasibility of the AA. We aim to identify true positive from true negative test performance between neglect cases (second consecutive sample) and a consecutive comparison group (neurological intact inpatients matched in age and gender). Sample size estimates for Study 5 are based on existing computer-based studies of neglect (Bonato, Priftis, Marenzi, Umiltà, & Zorzi, Reference Bonato, Priftis, Marenzi, Umiltà and Zorzi2010) Taking a conventional large effect size d = 0.8 (α = 0.05 and β = 0.80), the required sample size for this case-control study is N = 42 (21 patients and 21 controls). A large effect size is expected based on studies showing computer-based testing is sensitive and specific in detecting neglect (Bonato, Priftis, Marenzi, Umiltà, & Zorzi, Reference Bonato, Priftis, Marenzi, Umiltà and Zorzi2010, Reference Bonato, Priftis, Marenzi, Umiltà and Zorzi2012; Bonato et al., Reference Bonato, Priftis, Umilta and Zorzi2013). For our purposes, power analysis is based on between-samples t-test comparison of neglect versus controls; for example, the mean horizontal raycast position for neglect versus controls. Quantitative data will be analysed using descriptive statistics and ANOVAs for an independent-samples comparison.

Study six aims to enhance the task and experiential value of the AA. We will individually work with a small number of participants (N = 5 inpatients who are within 1 month of discharge from hospital) to enhance the platform with additional graphics and narrative context. After a demonstration test, we will seek their feedback and suggestions, and will incorporate video game approaches to guide the reward and engagement components of the task to ensure that future patients are motivated in the cognitive assessment platform. We will also work with the onsite clinical team to determine the value of the case report analyses and modifications for producing a clinically meaningful ‘fingerprint’ of the volumetric neglected space for individual patients, along with preliminary procedural steps for a clinician-operated mode (e.g., including a user-friendly graphical user interface). Aim three will establish the clinical value of the AA, accompanied by a full suite of testing options for a larger clinical trial and development into full game capability. We will engage the skills of an expert graphic narrator consultant who will identify and develop opportunities to enrich the task graphics and immersion.

Conclusions and next steps

The AA builds on the foundation of research into VR for detecting neglect. It aims to create a neglect detection VR platform that is highly sensitive, enjoyable, and effective for clinical use. The AA uses the latest technology, integrates eye-tracking, and will provide clinicians with immediate, clinically relevant feedback on a patient’s level of neglect. The research proposed is an iterative process where feedback from patients and clinicians will guide the next phase of the AA. This paper outlines why we have developed the current application, and our research plans to develop it further. Better assessment leads to better treatment; more enjoyable treatment leads to better outcomes. The AA applies the principles of translational neuroscience, whereby new generation cognitive assessment tools may ultimately be developed into VR games offering optimal scientific accuracy and patient engagement.

Supplementary materials

For supplementary material for this article, please visit https://doi.org/10.1017/BrImp.2022.15

Acknowledgements

Griffith University researchers and clinicians at Gold Coast University Hospital and Logan Hospital (Queensland, Australia) codesigned the VR game within The Hopkins Centre’s Brain and Enriched Environment (BEEhive) Laboratory for cognitive rehabilitation.

Financial support

The development work and research programme described in this paper was supported by a National Health and Medical Research Council Ideas Grant (APP2002362) 'DImensional Attention MOdelling for Neglect Detection (DIAMOND): A novel application for brain injury', a Metro South Health Research Support Scheme (MSH RSS) Project Grant (RSS_2021_173) '3D visuospatial attention mapping in patients with stroke and other neurological conditions' and First Prize in the Bionics Queensland Challenge 2020 in AI-Enabled Bionics 'AI-enabled spatial attention assessment and training system'.

DH is supported by an Early Career Research Fellowship from the National Health and Medical Research Council of Australia (GNT1142929).

Conflicts of interest

DSH receives royalties for books about pain and perception from Noigroup publications. He has also received travel support from Reality Health and speaking fees from professional and scientific bodies for lectures on pain.

All other authors have no conflicts of interest to declare that are relevant to the content of this article.

Ethical standards

The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008.

Open practices statement

The version of the AA software described in this paper is and will be publicly availability on GitHub (https://github.com/davidrosspainter/TheAttentionAtlas). Additionally, the software will be made publicly available on Open Science Framework (https://osf.io/pa96f/) prior to publication. The software will be made available under the CC BY-SA license, which allows reusers to distribute, remix, adapt, and build upon the material in any medium or format, so long as attribution is given to the creator. The license allows for commercial use and reuse under the same terms.

Data sharing not applicable to this article as no datasets were generated or analysed during the current study.

References

Adlakha, S., Chhabra, D., & Shukla, P. (2020). Effectiveness of gamification for the rehabilitation of neurodegenerative disorders. Chaos, Solitons and Fractals, 140, 110192. doi: 10.1016/j.chaos.2020.110192 CrossRefGoogle Scholar
Albert, M. L. (1973). A simple test of visual neglect. Neurology, 23(6), 658664. doi: 10.1212/WNL.23.6.658 CrossRefGoogle ScholarPubMed
Azouvi, P., Samuel, C., Louis-Dreyfus, A., Bernati, T., Bartolomeo, P., Beis, J., … French Collaborative Study Group on Assessment of Unilateral Neglect (GEREN/GRECO) (2002). Sensitivity of clinical and behavioural tests of spatial neglect after right hemisphere stroke. Journal of Neurology, Neurosurgery and Psychiatry, 73(2), 160166. doi: 10.1136/jnnp.73.2.160CrossRefGoogle ScholarPubMed
Bavelier, D., & Green, C. S. (2019). Enhancing attentional control: Lessons from action video games. Neuron, 104(1), 147163. doi: 10.1016/j.neuron.2019.09.031 CrossRefGoogle ScholarPubMed
Blascheck, T., Kurzhals, K., Raschke, M., Burch, M., Weiskopf, D., & Ertl, T. (2017). Visualization of eye tracking data: A taxonomy and survey: Visualization of eye tracking data. Computer Graphics Forum, 36(8), 260284. doi: 10.1111/cgf.13079 CrossRefGoogle Scholar
Bonato, M., Priftis, K., Marenzi, R., Umiltà, C., & Zorzi, M. (2010). Increased attentional demands impair contralesional space awareness following stroke. Neuropsychologia, 48(13), 39343940. doi: 10.1016/j.neuropsychologia.2010.08.022 CrossRefGoogle ScholarPubMed
Bonato, M., Priftis, K., Marenzi, R., Umiltà, C., & Zorzi, M. (2012). Deficits of contralesional awareness: A case study on what paper-and-pencil tests neglect. Neuropsychology, 26(1), 2036. doi: 10.1037/a0025306 CrossRefGoogle ScholarPubMed
Bonato, M., Priftis, K., Umilta, C., & Zorzi, M. (2013). Computer-based attention-demanding testing unveils severe neglect in apparently intact patients. Behavioural Neurology, 26(3), 179181. doi: 10.1155/2013/139812 CrossRefGoogle ScholarPubMed
Bowen, D. J., Kreuter, M., Spring, B., Cofta-Woerpel, L., Linnan, L., Weiner, D., …Fernandez, M. (2009). How we design feasibility studies. American Journal of Preventive Medicine, 36(5), 452457. doi: 10.1016/j.amepre.2009.02.002 CrossRefGoogle ScholarPubMed
Brooke, J. (1996). SUS: A ‘quick and dirty’ usability scale. In P. W. Jordan, B. Thomas, I. L. McClelland & B. Weerdmeester (Eds.), Useability evaluation in industry (pp. 189–194). London: Taylor Francis.Google Scholar
Burdea, G. C. (2003). Virtual rehabilitation-benefits and challenges. Methods of Information in Medicine, 42(05), 519523. doi: 10.1055/s-0038-1634378 Google ScholarPubMed
Buxbaum, L. J., Ferraro, M. K., Veramonti, T., Farne, A., Whyte, J., Ladavas, E., …Coslett, H. B. (2004). Hemispatial neglect: Subtypes, neuroanatomy, and disability. Neurology, 62(5), 749756. doi: 10.1212/01.WNL.0000113730.73031.F4 CrossRefGoogle ScholarPubMed
Buxbaum, L. J., Dawson, A. M., & Linsley, D. (2012). Reliability and validity of the virtual reality lateralized attention test in assessing hemispatial neglect in right-hemisphere stroke. Neuropsychology, 26(4), 430441. doi: 10.1037/a0028674 CrossRefGoogle ScholarPubMed
Castelvecchi, D. (2016). Low-cost headsets boost virtual reality’s lab appeal. Nature, 533(7602), 153154. doi: 10.1038/533153a CrossRefGoogle ScholarPubMed
Checketts, M., Mancuso, M., Fordell, H., Chen, P., Hreha, K., Eskes, G. A., …Bowen, A. (2020). Current clinical practice in the screening and diagnosis of spatial neglect post-stroke: Findings from a multidisciplinary international survey. Neuropsychological Rehabilitation, 31(9), 14951526. doi: 10.1080/09602011.2020.1782946,CrossRefGoogle ScholarPubMed
Committeri, G., Pitzalis, S., Galati, G., Patria, F., Pelle, G., Sabatini, U., …Pizzamiglio, L. (2007). Neural bases of personal and extrapersonal neglect in humans. Brain, 130(2), 431441. doi: 10.1093/brain/awl265CrossRefGoogle ScholarPubMed
Conti, R. P., & Arnone, J. M. (2016). Unilateral neglect: Assessment and rehabilitation. International Journal of Neuroscience and Behavioral Science, 4(1), 110. doi: 10.13189/ijnbs.2016.040101 CrossRefGoogle Scholar
Coyle, H., Traynor, V., & Solowij, N. (2015). Computerized and virtual reality cognitive training for individuals at high risk of cognitive decline: Systematic review of the literature. The American Journal of Geriatric Psychiatry, 23(4), 335359. doi: 10.1016/j.jagp.2014.04.009 CrossRefGoogle ScholarPubMed
Deterding, S., Dixon, D., Khaled, R., & Nacke, L. (2011). From game design elements to gamefulness: Defining “gamification”. Proceedings of the 15th International Academic MindTrek Conference on Envisioning Future Media Environments MindTrek ’11, 9. doi: 10.1145/2181037.2181040 CrossRefGoogle Scholar
Dvorkin, A. Y., Bogey, R. A., Harvey, R. L., & Patton, J. L. (2012, February). Mapping the neglected space: gradients of detection revealed by virtual reality. Neurorehabil Neural Repair, 26(2), 120131. doi: 10.1177/1545968311410068 CrossRefGoogle ScholarPubMed
Farrow, S., & Reid, D. (2004). Stroke survivors' perceptions of a leisure-based virtual reality program. Technology and Disability, 16(2), 6981. doi: 10.3233/tad-2004-16201 CrossRefGoogle Scholar
Fordell, H., Bodin, K., Bucht, G., & Malm, J. (2011). A virtual reality test battery for assessment and screening of spatial neglect. Acta Neurologica Scandinavica, 123(3), 167174. doi: 10.1111/j.1600-0404.2010.01390.x CrossRefGoogle ScholarPubMed
Freedman, M., Leach, L., Kaplan, E., Shulman, K., & Delis, D. C. (1994). Clock drawing: A neuropsychological analysis. New York: Oxford University Press.Google Scholar
Friedrich, T. E., Hunter, P. V., & Elias, L. J. (2018). The trajectory of pseudoneglect in adults: A systematic review. Neuropsychology Review, 28(4), 436452. doi: 10.1007/s11065-018-9392-6 CrossRefGoogle ScholarPubMed
Freitas, L., de Araújo Val, S., Magalhaes, F., Marinho, V., Ayres, C., Teixeira, S., & Bastos, V. H. (2021). Virtual reality exposure therapy for neuro-psychomotor recovery in adults: A systematic review. Disability and Rehabilitation: Assistive Technology, 16(6), 646652. doi: 10.1080/17483107.2019.1688400 Google ScholarPubMed
Gauthier, L., Dehaut, F., & Joanette, Y. (1989). The bells test: A quantitative and qualitative test for visual neglect. International Journal of Clinical Neuropsychology, 11(2), 4954.Google Scholar
Gorman, C., & Gustafsson, L. (2020). The use of augmented reality for rehabilitation after stroke: A narrative review. Disability and Rehabilitation: Assistive Technology, 19. doi: 10.1080/17483107.2020.1791264 Google ScholarPubMed
Gorman, C., Gustafsson, L., & Gomura, C. (2021). The perspectives of stroke survivors and health professionals on the use of augmented reality for inpatient stroke rehabilitation: An anticipatory exploration. Brain Impairment, 110. doi: 10.1017/BrImp.2021.11CrossRefGoogle Scholar
Halligan, P., & Marshall, J. (1989). is neglect (only) lateral a quadrant analysis of line cancellation. Journal of Clinical and Experimental Neuropsychology, 11(6), 793798. doi: 10.1080/01688638908400936 CrossRefGoogle ScholarPubMed
Halligan, P., & Marshall, J. (1991). Left neglect for near but not far space in man. Nature, 350, 498500. doi: 10.1038/350498a0 CrossRefGoogle Scholar
Harada, Y., & Ohyama, J. (2019, November 6). Spatiotemporal characteristics of 360-degree basic attention. Scientific Reports, 9(1), 16083. doi: 10.1038/s41598-019-52313-3 CrossRefGoogle ScholarPubMed
Harvey, M., Learmonth, G., Rossit, S., & Chen, P. (2021). Editorial for special issue on neglect rehabilitation. Neuropsychological Rehabilitation, 110. doi: 10.1080/09602011.2021.1873150 CrossRefGoogle ScholarPubMed
He, B. J., Snyder, A. Z., Vincent, J. L., Epstein, A., Shulman, G. L., & Corbetta, M. (2007). Breakdown of functional connectivity in frontoparietal networks underlies behavioral deficits in spatial neglect. Neuron, 53(6), 905918. doi: 10.1016/j.neuron.2007.02.013 CrossRefGoogle ScholarPubMed
Henry, D., Dymnicki, A. B., Mohatt, N., Allen, J., & Kelly, J. G. (2015). Clustering methods with qualitative data: A mixed-methods approach for prevention research with small samples. Prevention Science, 16(7), 10071016. doi: 10.1007/s11121-015-0561-z CrossRefGoogle ScholarPubMed
Howard, C., & Rowe, F. J. (2018). Adaptation to poststroke visual field loss: A systematic review. Brain and Behavior, 8(8), e01041. doi: 10.1002/brb3.1041 CrossRefGoogle ScholarPubMed
Huygelier, H., Mattheus, E., Vanden Abeele, V., Van Ee, R., & Gillebert, C. R. (2021). The use of the term virtual reality in post-stroke rehabilitation: A scoping review and commentary. Psychologica Belgica, 61(1), 145162. doi: 10.5334/pb.1033 CrossRefGoogle ScholarPubMed
Jewell, G., & McCourt, M. E. (2000). Pseudoneglect: A review and meta-analysis of performance factors in line bisection tasks. Neuropsychologia, 38(1), 93110. doi: 10.1016/S0028-3932(99)00045-7 CrossRefGoogle ScholarPubMed
Kaufmann, B. C., Cazzoli, D., Muri, R. M., Nef, T., & Nyffeler, T. (2020a). Test-retest-reliability of video-oculography during free visual exploration in right-hemispheric stroke patients with neglect. Frontiers in Neuroscience, 14, 731. doi: 10.3389/fnins.2020.00731 CrossRefGoogle ScholarPubMed
Kaufmann, B. C., Cazzoli, D., Pflugshaupt, T., Bohlhalter, S., Vanbellingen, T., Muri, R. M., …Nyffeler, T. (2020b). Eyetracking during free visual exploration detects neglect more reliably than paper-pencil tests. Cortex, 129, 223235. doi: 10.1016/j.cortex.2020.04.021 CrossRefGoogle ScholarPubMed
Kennedy, R. S., Lane, N. E., Berbaum, K. S., & Lilienthal, M. G. (1993). Simulator sickness questionnaire: An enhanced method for quantifying simulator sickness. The International Journal of Aviation Psychology, 3(3), 203220. doi: 10.1207/s15327108ijap0303_3 CrossRefGoogle Scholar
Keshavarz, B., Hecht, H., & Zschutschke, L. (2011). Intra-visual conflict in visually induced motion sickness. Displays, 32(4), 181188. doi: 10.1016/j.displa.2011.05.009 CrossRefGoogle Scholar
Kim, J., Kim, K., Kim, D. Y., Chang, W. H., Park, C., Ohn, S. H., …Kim, S. I. (2007). Virtual environment training system for rehabilitation of stroke patients with unilateral neglect: Crossing the virtual street. Cyberpsychology & Behavior, 10(1), 715. doi: 10.1089/cpb.2006.9998 CrossRefGoogle ScholarPubMed
Kim, D. Y., Ku, J., Chang, W. H., Park, T. H., Lim, J. Y., Han, K., …Kim, S. I. (2010). Assessment of post-stroke extrapersonal neglect using a three-dimensional immersive virtual street crossing program. Acta Neurologica Scandinavica, 121(3), 171177. doi: 10.1111/j.1600-0404.2009.01194.x CrossRefGoogle ScholarPubMed
Knobel, S. E. J., Kaufmann, B. C., Gerber, S. M., Cazzoli, D., Muri, R. M., Nyffeler, T., & Nef, T. (2020). Immersive 3D virtual reality cancellation task for visual neglect assessment: A pilot study. Frontiers in Human Neuroscience, 14, 180. doi: 10.3389/fnhum.2020.00180 CrossRefGoogle ScholarPubMed
Lange, B., Flynn, S., Proffitt, R., Chang, C., & Rizzo, A. S. (2010). Development of an interactive game-based rehabilitation tool for dynamic balance training. Topics in Stroke Rehabilitation, 17(5), 345352. doi: 10.1310/tsr1705-345 CrossRefGoogle ScholarPubMed
Laver, K. E., Adey-Wakeling, Z., Crotty, M., Lannin, N. A., George, S., & Sherrington, C. (2020). Telerehabilitation services for stroke. Cochrane Library, 2020(1), CD010255. doi: 10.1002/14651858.CD010255.pub3 Google Scholar
Laver, K. E., Lange, B., George, S., Deutsch, J. E., Saposnik, G., & Crotty, M. (2017). Virtual reality for stroke rehabilitation. Cochrane Database of Systematic Reviews, 11(1), 173.Google ScholarPubMed
Lohse, K. R., Hilderman, C. G. E., Cheung, K. L., Tatla, S., & Van Der Loos, H. F. M. (2014). Virtual reality therapy for adults post-stroke: A systematic review and meta-analysis exploring virtual environments and commercial games in therapy. PloS One, 9(3), e93318. doi: 10.1371/journal.pone.0093318 CrossRefGoogle ScholarPubMed
Longley, V., Hazelton, C., Heal, C., Pollock, A., Woodward-Nutt, K., Mitchell, C., …Bowen, A. (2021). Non-pharmacological interventions for spatial neglect or inattention following stroke and other non-progressive brain injury. Cochrane Database of Systematic Reviews, 7, CD003586. doi: 10.1002/14651858.CD003586.pub4,Google ScholarPubMed
Morse, H., Biggart, L., Pomeroy, V., & Rossit, S. (2020). Exploring perspectives from stroke survivors, carers and clinicians on virtual reality as a precursor to using telerehabilitation for spatial neglect post-stroke. Neuropsychological Rehabilitation, 125. doi: 10.1080/09602011.2020.1819827 CrossRefGoogle ScholarPubMed
Nijboer, T. C. W., Kollen, B. J., & Kwakkel, G. (2013). Time course of visuospatial neglect early after stroke: A longitudinal cohort study. Cortex, 49(8), 20212027. doi: 10.1016/j.cortex.2012.11.006 CrossRefGoogle ScholarPubMed
Ogourtsova, T., Souza Silva, W., Archambault, P. S., & Lamontagne, A. (2017, April). Virtual reality treatment and assessments for post-stroke unilateral spatial neglect: A systematic literature review. Neuropsychological Rehabilitation, 27(3), 409454. doi: 10.1080/09602011.2015.1113187 CrossRefGoogle ScholarPubMed
Ogourtsova, T., Archambault, P. S., & Lamontagne, A. (2019, February). Exploring barriers and facilitators to the clinical use of virtual reality for post-stroke unilateral spatial neglect assessment. Disability and Rehabilitation, 41(3), 284292. doi: 10.1080/09638288.2017.1387292,CrossRefGoogle Scholar
Painter, D. R., Dux, P. E., Travis, S. L., & Mattingley, J. B. (2014). Neural responses to target features outside a search array are enhanced during conjunction but not unique-feature search. Journal of Neuroscience, 34(9), 33903401.CrossRefGoogle Scholar
Paolucci, S., Antonucci, G., Grasso, M. G., & Pizzamiglio, L. (2001). The role of unilateral spatial neglect in rehabilitation of right brain-damaged ischemic stroke patients: A matched comparison. Archives of Physical Medicine and Rehabilitation, 82(6), 743749. doi: 10.1053/apmr.2001.23191 CrossRefGoogle ScholarPubMed
Parker, A. M., Lord, R. K., & Needham, D. M. (2013). Increasing the dose of acute rehabilitation: Is there a benefit? BMC Medicine, 11(1), 199199. doi: 10.1186/1741-7015-11-199 CrossRefGoogle ScholarPubMed
Pedrazzini, E., & Ptak, R. (2020). The neuroanatomy of spatial awareness: A large-scale region-of-interest and voxel-based anatomical study. Brain Imaging and Behavior, 14(2), 615626. doi: 10.1007/s11682-019-00213-5 CrossRefGoogle ScholarPubMed
Pedroli, E., Serino, S., Cipresso, P., Pallavicini, F., & Riva, G. (2015, August). Assessment and rehabilitation of neglect using virtual reality: A systematic review. Frontiers in Behavioral Neuroscience, 9, 226. doi: 10.3389/fnbeh.2015.00226 CrossRefGoogle ScholarPubMed
Pietrzak, E., Pullman, S., & McGuire, A. (2014). Using virtual reality and videogames for traumatic brain injury rehabilitation: A structured literature review. Games for Health, 3(4), 22214. doi: 10.1089/g4h.2014.0013 Google ScholarPubMed
Poels, K., de Kort, Y. A. W., & IJsselsteijn, W. A. (2007). D3.3: Game experience questionnaire: Development of a self-report measure to assess the psychological impact of digital games. Eindhoven: Technische Universiteit Eindhoven.Google Scholar
Ribolsi, M., Di Lorenzo, G., Lisi, G., Niolu, C., & Siracusano, A. (2015). A critical review and meta-analysis of the perceptual pseudoneglect across psychiatric disorders: Is there a continuum? Cognitive Processing, 16(1), 1725. doi: 10.1007/s10339-014-0640-2 CrossRefGoogle ScholarPubMed
Ringman, J. M., Saver, J. L., Woolson, R. F., Clarke, W. R., & Adams, H. P. (2004). Frequency risk factors, anatomy, and course of unilateral neglect in an acute stroke cohort. Neurology, 63(3), 468474. doi: 10.1212/01.WNL.0000133011.10689.CE CrossRefGoogle Scholar
Santana, M., Zelinsky, S., Ahmed, S., Doktorchik, C., James, M., Wilton, S., …Butalia, S. (2020). Patients, clinicians and researchers working together to improve cardiovascular health: A qualitative study of barriers and priorities for patient-oriented research. BMJ Open, 10(2), e031187. doi: 10.1136/bmjopen-2019-031187 CrossRefGoogle ScholarPubMed
Shah, N., Basteris, A., & Amirabdollahian, F. (2014). Design parameters in multimodal games for rehabilitation. Games for Health Journal, 3(1), 1320. doi: 10.1089/g4h.2013.0044 CrossRefGoogle ScholarPubMed
Syed, U. E., & Kamal, A. (2021). Video game-based and conventional therapies in patients of neurological deficits: An experimental study. Disability and Rehabilitation: Assistive Technology, 16(3), 332339. doi: 10.1080/17483107.2019.1679266 Google ScholarPubMed
Tavaszi, I., Nagy, A. S., Szabo, G., & Fazekas, G. (2020/2021). Neglect syndrome in post-stroke conditions: Assessment and treatment (scoping review). International Journal of Rehabilitation Research, 44(1), 314. doi: 10.1097/MRR.0000000000000438 CrossRefGoogle Scholar
Thornton, M., Marshall, S., McComas, J., Finestone, H., McCormick, A., & Sveistrup, H. (2005). Benefits of activity and virtual reality based balance exercise programmes for adults with traumatic brain injury: Perceptions of participants and their caregivers. Brain Injury, 19(12), 9891000. doi: 10.1080/02699050500109944 CrossRefGoogle ScholarPubMed
Threapleton, K., Drummond, A., & Standen, P. (2016). Virtual rehabilitation: What are the practical barriers for home-based research? Digital Health, 2, 2055207616641302. doi: 10.1177/2055207616641302 CrossRefGoogle ScholarPubMed
Umeonwuka, C., Roos, R., & Ntsiea, V. (2020). Current trends in the treatment of patients with post-stroke unilateral spatial neglect: A scoping review. Disability and Rehabilitation, 128. doi: 10.1080/09638288.2020.1824026 CrossRefGoogle ScholarPubMed
Vuilleumier, P. (2013). Mapping the functional neuroanatomy of spatial neglect and human parietal lobe functions: Progress and challenges. Annals of the New York Academy of Sciences, 1296(1), 5074. doi: 10.1111/nyas.12161 CrossRefGoogle ScholarPubMed
Wee, J. Y. M., & Hopman, W. M. (2008). Comparing consequences of right and left unilateral neglect in a stroke rehabilitation population. American Journal of Physical Medicine & Rehabilitation, 87(11), 910920. doi: 10.1097/PHM.0b013e31818a58bd CrossRefGoogle Scholar
Wei, P., Yu, H., Müller, H. J., Pollmann, S., & Zhou, X. (2019). Differential brain mechanisms for processing distracting information in task-relevant and -irrelevant dimensions in visual search. Human Brain Mapping, 40(1), 110124. doi: 10.1002/hbm.24358 CrossRefGoogle ScholarPubMed
Wolfe, J. M. (1998). What can 1 million trials tell us about visual search? Psychological Science, 9(1), 3339. doi: 10.1111/1467-9280.00006 CrossRefGoogle Scholar
Wolfe, J. M., & Horowitz, T. S. (2004). What attributes guide the deployment of visual attention and how do they do it? Nature Reviews Neuroscience, 5(6), 495501. doi: 10.1038/nrn1411 CrossRefGoogle ScholarPubMed
Wu, Y., Wang, J., Zhang, Y., Zheng, D., Zhang, J., Rong, M., …Jiang, T. (2016). The neuroanatomical basis for posterior superior parietal lobule control lateralization of visuospatial attention. Frontiers in Neuroanatomy, 10, 3232. doi: 10.3389/fnana.2016.00032 CrossRefGoogle ScholarPubMed
Zeeman, H. (2013). Systems for healing: Building a better service system for traumatic brain injury. In Zeeman, H., Kendall, E. & Wright, J. J. (Eds.), Health and healing after brain injury: Understanding the power of family, friends, community and other support systems (section 3). Westport, CT: Praeger.Google Scholar
Figure 0

Figure 1. The attention atlas. Note. This diagram depicts various search arrays with varying parameters and an icosphere raycast surface positioned with a radius of 1.5 m with respect to the origin (headset position). (a) Trial structure. Cues and arrays are presented until they are selected by the user via pointing and a button press. (b) Visual search paradigm and (c) raycast surface and sources viewed from the third person perspective. The raycast surface is an icosphere (recursion [i.e., subdivision] level = 3) mesh defined by triangular faces and vertices located at the intersections of edges. During the game, the raycast surface is invisible. (d) Raycast surface viewed from a variety of angles. (e) Coordinate systems for array element positioning. (f) Depth configurations. (g) Search modes based on colour variation. (h) Preconfigured stimulus options. Arrays in panels b, e–f have an element inclusion angle of 50.0°. Arrays in panels g–h have an element inclusion angle of 12.5°. Panels e–h are viewed from the first-person perspective. All materials are publicly available: the X bot character model depicted is from Mixamo (https://www.mixamo.com/#/?page=3&type=Character); letters and numbers are from the OpenDyslexic typeface (https://opendyslexic.org/); Georgian symbols are from i2symbol (https://www.i2symbol.com/abc-123/georgian); shapes and balloons are custom-made; playing cards are from Google (https://code.google.com/archive/p/vector-playing-cards/).

Figure 1

Figure 2. Instructions for players and element selection. Note. The player indicates that they have found the target by pointing using the controller and pressing the thumb button.

Figure 2

Figure 3. Origin calibration. Note. (a) Origin targets and VR camera rig shown from a top-down perspective. (b) Origin target raycast depicted from the first-person VR perspective.

Figure 3

Figure 4. Game and level progression. Note. (a) This example game is based on the spherical coordinate system. Black scatter points reflect array element positions, with a minimal and even spacing of 12.5° on latitude and longitude axes. (b) Example level progression. The game starts with a tutorial and then progresses through a series of four levels, with later levels tending to be more difficult than those proceeding. Each level is comprised of one or more conditions that can be presented in a random order.

Figure 4

Figure 5. Near-time attention maps and behavioural performance. Note. (a) Game start time. (b) Level descriptor. Each level is plotted with all conditions combined (shown here) and for each condition separately (not shown). (c) Raycast scatterplot of raw (x, y, z) data converted to spherical coordinates. The results are presented from the player’s forward-facing perspective. Red reflects headset raycasts, blue reflects controller raycasts, and black reflects eye gaze raycasts. (d) Target detection performance (accuracy and RT). (e) System performance. (f–h) Trial counts and overall behavioural performance as a function of target position. (f) Trials count by target position map. Whiter shades reflect higher counts. (g) Accuracy by target position map. Whiter shades reflect higher accuracy. (h) RT by target position map. Whiter shades reflect faster RTs. (i-k) Raycast heatmaps (2D histograms) for each attentional source. Brighter colours reflect more frequently attended locations. Horizontal and vertical bin size is set to 1°. (i) Raycast headset heatmap. (j) Raycast controller heatmap. (k) Raycast gaze heatmap. (l–n) Latitudinal attentional distribution and symmetry tests for the raycast sources (headset: l, controller: m, gaze: n). Raycast positions are represented as kernel density functions (upper subplots), with inlaid boxplots reflecting the median, interquartile range (IQR), and 1.5 × IQR (whiskers). The significance test for latitudinal symmetry (i.e., a mean central attention not significantly different from zero; lower subplots), is a one sample permutation test, with the sign of latitudinal raycasts randomised on each of N = 100 permutations (α = 0.05, two-tailed). The blue kernel density function reflects the permutation distribution, and the red line indicates the mean obtained value. Significance values are shown in the subplot title, with p < 0.05 indicating significant asymmetries. (o–q) Longitudinal attentional distribution and symmetry tests for the raycast sources (headset: o, controller: p, gaze: q). The procedure is identical to the latitude tests, and the orientation of the plots has been switched to depict vertical attention. (r) Level options. The stimulus configuration that produced the results is reported for evaluation and reference.

Figure 5

Table 1. The Key Game Classes

Figure 6

Table 2. The Key Analyser Modules

Figure 7

Figure 6. Data and results folder and file structure. Note: (a) Game data structure. (b) Level data structure. (c) Game results structure. (d) Level results structure.

Figure 8

Table 3. Aims and Studies

Supplementary material: File

Norwood et al. supplementary material

Norwood et al. supplementary material

Download Norwood et al. supplementary material(File)
File 18 KB
You have Access Open access

Save article to Kindle

To save this article to your Kindle, first ensure coreplatform@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

The attention atlas virtual reality platform maps three-dimensional (3D) attention in unilateral spatial neglect patients: a protocol
Available formats
×

Save article to Dropbox

To save this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you used this feature, you will be asked to authorise Cambridge Core to connect with your Dropbox account. Find out more about saving content to Dropbox.

The attention atlas virtual reality platform maps three-dimensional (3D) attention in unilateral spatial neglect patients: a protocol
Available formats
×

Save article to Google Drive

To save this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you used this feature, you will be asked to authorise Cambridge Core to connect with your Google Drive account. Find out more about saving content to Google Drive.

The attention atlas virtual reality platform maps three-dimensional (3D) attention in unilateral spatial neglect patients: a protocol
Available formats
×
×

Reply to: Submit a response

Please enter your response.

Your details

Please enter a valid email address.

Conflicting interests

Do you have any conflicting interests? *