Hostname: page-component-848d4c4894-75dct Total loading time: 0 Render date: 2024-05-13T21:29:42.825Z Has data issue: false hasContentIssue false
  • English
  • Français

DEMONSTRATING THE FEASIBILITY OF MULTIMODAL NEUROIMAGING DATA CAPTURE WITH A WEARABLE ELECTOENCEPHALOGRAPHY + FUNCTIONAL NEAR-INFRARED SPECTROSCOPY (EEG+FNIRS) IN SITU

Published online by Cambridge University Press:  27 July 2021

Henrikke Dybvik ,
Christian Kuster Erichsen  and
Martin Steinert
Henrikke Dybvik*
Affiliation:
Norwegian University of Science and Technology
Christian Kuster Erichsen
Affiliation:
Norwegian University of Science and Technology
Martin Steinert
Affiliation:
Norwegian University of Science and Technology
*
Dybvik, Henrikke, Norwegian University of Science and Technology, Department of Mechanical and Industrial Engineering, Norway, henrikke.dybvik@ntnu.no

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

We developed a wearable experimental sensor setup featuring multimodal EEG+fNIRS neuroimaging applicable for in situ experiments of human behavior in interaction with technology. A low-cost electroencephalography (EEG) was integrated with a wearable functional Near-Infrared Spectroscopy (fNIRS) system, which we present in two parts. Paper A provide an exhaustive description of setup infrastructure, data synchronization process, a procedure for usage, including sensor application, and ensuring high signal quality. This paper (Paper B) demonstrate the setup';s usability in three distinct use cases: a conventional human-computer interaction experiment, an in situ driving experiment where participants drive a car in the city and on the highway, and an ashtanga vinyasa yoga practice in situ. Data on cognitive load from highly ecologically valid experimental setups are presented, and we discuss lessons learned. These include acceptable and unacceptable artefacts, data quality, and constructs possible to investigate with the setup.

Keywords

EEG+fNIRSin situ experimentsHuman behaviour in designUser centred designResearch methodologies and methods
Type
Article
Information
Proceedings of the Design Society , Volume 1: ICED21 , August 2021 , pp. 901 - 910
Creative Commons
Creative Common License - CCCreative Common License - BYCreative Common License - NCCreative Common License - ND
This is an Open Access article, distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives licence (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is unaltered and is properly cited. The written permission of Cambridge University Press must be obtained for commercial re-use or in order to create a derivative work.
Copyright
The Author(s), 2021. Published by Cambridge University Press

References

Ahn, S., & Jun, S. C. (2017). Multi-Modal Integration of EEG-fNIRS for Brain-Computer Interfaces – Current Limitations and Future Directions. Frontiers in Human Neuroscience, 11. https://doi.org/10.3389/fnhum.2017.00503CrossRefGoogle ScholarPubMed
Ahn, S., Nguyen, T., Jang, H., Kim, J. G., & Jun, S. C. (2016). Exploring Neuro-Physiological Correlates of Drivers’ Mental Fatigue Caused by Sleep Deprivation Using Simultaneous EEG, ECG, and fNIRS Data. Frontiers in Human Neuroscience, 10. https://doi.org/10.3389/fnhum.2016.00219CrossRefGoogle ScholarPubMed
Antonenko, P., Paas, F., Grabner, R., & van Gog, T. (2010). Using Electroencephalography to Measure Cognitive Load. Educational Psychology Review, 22(4), 425–438. https://doi.org/10.1007/s10648-010-9130-yCrossRefGoogle Scholar
Ashtanga Yoga Full Primary Series with Ty Landrum. (2020). https://www.youtube.com/watch?v=K-s4IIxVBc8&t=2607sGoogle Scholar
Balters, S., & Steinert, M. (2017). Capturing emotion reactivity through physiology measurement as a foundation for affective engineering in engineering design science and engineering practices. Journal of Intelligent Manufacturing, 28(7), 1585–1607. https://doi.org/10.1007/s10845-015-1145-2CrossRefGoogle Scholar
Barker, J. W., Aarabi, A., & Huppert, T. J. (2013). Autoregressive model based algorithm for correcting motion and serially correlated errors in fNIRS. Biomedical Optics Express, 4(8), 1366. https://doi.org/10.1364/BOE.4.001366CrossRefGoogle ScholarPubMed
Borghini, G., Astolfi, L., Vecchiato, G., Mattia, D., & Babiloni, F. (2014). Measuring neurophysiological signals in aircraft pilots and car drivers for the assessment of mental workload, fatigue and drowsiness. Neuroscience & Biobehavioral Reviews, 44, 58–75. https://doi.org/10.1016/j.neubiorev.2012.10.003CrossRefGoogle ScholarPubMed
Cairns, P. E., & Cox, A. L. (2008). Research methods for human-computer interaction. Cambridge University Press.CrossRefGoogle Scholar
Cisler, D., Greenwood, P. M., Roberts, D. M., McKendrick, R., & Baldwin, C. L. (2019). Comparing the Relative Strengths of EEG and Low-Cost Physiological Devices in Modeling Attention Allocation in Semiautonomous Vehicles. Frontiers in Human Neuroscience, 13. https://doi.org/10.3389/fnhum.2019.00109CrossRefGoogle Scholar
Consolvo, S., Harrison, B., Smith, I., Chen, M. Y., Everitt, K., Froehlich, J., & Landay, J. A. (2007). Conducting In Situ Evaluations for and With Ubiquitous Computing Technologies. International Journal of Human–Computer Interaction, 22(1-2), 103–118. https://doi.org/10.1080/10447310709336957CrossRefGoogle Scholar
Delorme, A., & Makeig, S. (2004). EEGLAB: An open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. Journal of Neuroscience Methods, 134(1), 9–21. https://doi.org/10.1016/j.jneumeth.2003.10.009CrossRefGoogle ScholarPubMed
Dybvik, H., Erichsen, C. K., & Steinert, M. (Under Review). DESCRIPTION OF A WEARABLE ELECTROENCEPHALOGRAPHY AND FUNCTIONAL NEAR-INFRARED SPECTROSCOPY (EEG+FNIRS) FOR IN SITU EXPERIMENTS ON DESIGN COGNITION. Proceedings of the Design Society: International Conference on Engineering Design.Google Scholar
Fishburn, F. A., Ludlum, R. S., Vaidya, C. J., & Medvedev, A. V. (2019). Temporal Derivative Distribution Repair (TDDR): A motion correction method for fNIRS. NeuroImage, 184, 171–179. https://doi.org/10.1016/j.neuroimage.2018.09.025CrossRefGoogle ScholarPubMed
Gero, J. S., & Milovanovic, J. (2020). A framework for studying design thinking through measuring designers’ minds, bodies and brains. Design Science, 6, e19. https://doi.org/10.1017/dsj.2020.15CrossRefGoogle Scholar
Goucher-Lambert, K., Moss, J., & Cagan, J. (2019). A neuroimaging investigation of design ideation with and without inspirational stimuli—Understanding the meaning of near and far stimuli. Design Studies, 60, 1–38. https://doi.org/10.1016/j.destud.2018.07.001CrossRefGoogle Scholar
Hay, L., Cash, P., & McKilligan, S. (2020). The future of design cognition analysis. Design Science, 6. https://doi.org/10.1017/dsj.2020.20CrossRefGoogle Scholar
Herold, F., Wiegel, P., Scholkmann, F., & Müller, N. G. (2018). Applications of Functional Near-Infrared Spectroscopy (fNIRS) Neuroimaging in Exercise–Cognition Science: A Systematic, Methodology-Focused Review. Journal of Clinical Medicine, 7(12), 466. https://doi.org/10.3390/jcm7120466CrossRefGoogle Scholar
Ingre, M., Åkerstedt, T., Peters, B., Anund, A., & Kecklund, G. (2006). Subjective sleepiness, simulated driving performance and blink duration: Examining individual differences. Journal of Sleep Research, 15(1), 47–53. https://doi.org/10.1111/j.1365-2869.2006.00504.xCrossRefGoogle ScholarPubMed
Jacko, J. A. (2012). The human-computer interaction handbook: Fundamentals, evolving technologies, and emerging applications (3rd ed.). CRC Press.CrossRefGoogle Scholar
Jung, T.-P., Makeig, S., Humphries, C., Lee, T.-W., McKeown, M. J., Iragui, V., & Sejnowski, T. J. (2000). Removing electroencephalographic artifacts by blind source separation. Psychophysiology, 37(2), 163–178. https://doi.org/10.1111/1469-8986.3720163CrossRefGoogle ScholarPubMed
Kim, D.-W., & Im, C.-H. (2018). EEG Spectral Analysis. In Im, C.-H. (Ed.), Computational EEG Analysis: Methods and Applications (pp. 35–53). Springer. https://doi.org/10.1007/978-981-13-0908-3_3CrossRefGoogle Scholar
Klimesch, W. (1999). EEG alpha and theta oscillations reflect cognitive and memory performance: A review and analysis. Brain Research Reviews, 29(2), 169–195. https://doi.org/10.1016/S0165-0173(98)00056-3CrossRefGoogle ScholarPubMed
Lal, S. K. L., & Craig, A. (2001). A critical review of the psychophysiology of driver fatigue. Biological Psychology, 55(3), 173–194. https://doi.org/10.1016/S0301-0511(00)00085-5CrossRefGoogle ScholarPubMed
Mayseless, N., Hawthorne, G., & Reiss, A. L. (2019). Real-life creative problem solving in teams: FNIRS based hyperscanning study. NeuroImage, 203, 116161. https://doi.org/10.1016/j.neuroimage.2019.116161CrossRefGoogle ScholarPubMed
Mikkonen, J., Pedersen, P., & McCarthy, P. W. (2008). A Survey of Musculoskeletal Injury among Ashtanga Vinyasa Yoga Practitioners. International Journal of Yoga Therapy, 18(1), 59–64. https://doi.org/10.17761/ijyt.18.1.l0748p25k2558v77CrossRefGoogle Scholar
Okamoto, M., Dan, H., Shimizu, K., Takeo, K., Amita, T., Oda, I., Konishi, I., Sakamoto, K., Isobe, S., Suzuki, T., Kohyama, K., & Dan, I. (2004). Multimodal assessment of cortical activation during apple peeling by NIRS and fMRI. NeuroImage, 21(4), 1275–1288. https://doi.org/10.1016/j.neuroimage.2003.12.003CrossRefGoogle ScholarPubMed
Oostenveld, R., & Praamstra, P. (2001). The five percent electrode system for high-resolution EEG and ERP measurements. Clinical Neurophysiology, 112(4), 713–719. https://doi.org/10.1016/S1388-2457(00)00527-7CrossRefGoogle ScholarPubMed
Pinti, P., Tachtsidis, I., Hamilton, A., Hirsch, J., Aichelburg, C., Gilbert, S., & Burgess, P. W. (2018). The present and future use of functional near-infrared spectroscopy (fNIRS) for cognitive neuroscience. Annals of the New York Academy of Sciences. http://doi.org/10.1111/nyas.13948CrossRefGoogle Scholar
Piper, S. K., Krueger, A., Koch, S. P., Mehnert, J., Habermehl, C., Steinbrink, J., Obrig, H., & Schmitz, C. H. (2014). A Wearable Multi-Channel fNIRS System for Brain Imaging in Freely Moving Subjects. NeuroImage, 85(01). https://doi.org/10.1016/j.neuroimage.2013.06.062CrossRefGoogle ScholarPubMed
Santosa, H., Zhai, X., Fishburn, F., & Huppert, T. (2018). The NIRS Brain AnalyzIR Toolbox. Algorithms, 11(5), 73. https://doi.org/10.3390/a11050073CrossRefGoogle Scholar
Steinert, M., & Jablokow, K. (2013). Triangulating front end engineering design activities with physiology data and psychological preferences. 109–118.Google Scholar
Sterman, M., Mann, C., & Kaiser, D. (1993). Quantitative EEG patterns of differential in-flight workload.Google Scholar
Zeng, H., Song, A., Yan, R., & Qin, H. (2013). EOG Artifact Correction from EEG Recording Using Stationary Subspace Analysis and Empirical Mode Decomposition. Sensors (Basel, Switzerland), 13(11), 14839–14859. https://doi.org/10.3390/s131114839CrossRefGoogle Scholar