Hostname: page-component-8448b6f56d-xtgtn Total loading time: 0 Render date: 2024-04-23T14:13:46.545Z Has data issue: false hasContentIssue false
  • English
  • Français

DESCRIPTION OF A WEARABLE ELECTROENCEPHALOGRAPHY + FUNCTIONAL NEAR-INFRARED SPECTROSCOPY (EEG+FNIRS) FOR IN-SITU EXPERIMENTS ON DESIGN COGNITION

Published online by Cambridge University Press:  27 July 2021

Henrikke Dybvik ,
Christian Kuster Erichsen  and
Martin Steinert
Henrikke Dybvik*
Affiliation:
Norwegian University of Science and Technology
Christian Kuster Erichsen
Affiliation:
Norwegian University of Science and Technology
Martin Steinert
Affiliation:
Norwegian University of Science and Technology
*
Dybvik, Henrikke, Norwegian University of Science and Technology, Department of Mechanical and Industrial Engineering, Norway, henrikke.dybvik@ntnu.no

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

We developed a wearable experimental sensor setup featuring multimodal EEG+fNIRS neuroimaging data capture applicable for in situ experiments at a lower financial threshold. Consistent application of a good protocol and procedure for sensor application and signal quality control is crucial for researchers to obtain valid data. This paper provides an exhaustive description of the sensor setup, the data synchronization process, procedure for sensor application, and signal quality control. Potential design cognition experiments with the proposed EEG+fNIRS are also described. In summary, the setup is mobile and provides multimodal neuroimaging data of high quality. We encourage the design community to take advantage of the setup and adapt it to new experimental setups in situ.

Keywords

EEG+fNIRSMobile ExperimentsHuman behaviour in designDesign cognitionResearch methodologies and methods
Type
Article
Information
Proceedings of the Design Society , Volume 1: ICED21 , August 2021 , pp. 943 - 952
Creative Commons
Creative Common License - CCCreative Common License - BYCreative Common License - NCCreative Common License - ND
This is an Open Access article, distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives licence (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is unaltered and is properly cited. The written permission of Cambridge University Press must be obtained for commercial re-use or in order to create a derivative work.
Copyright
The Author(s), 2021. Published by Cambridge University Press

References

Ahn, S., & Jun, S. C. (2017). Multi-Modal Integration of EEG-fNIRS for Brain-Computer Interfaces – Current Limitations and Future Directions. Frontiers in Human Neuroscience, 11. https://doi.org/10.3389/fnhum.2017.00503CrossRefGoogle ScholarPubMed
Ahn, S., Nguyen, T., Jang, H., Kim, J. G., & Jun, S. C. (2016). Exploring Neuro-Physiological Correlates of Drivers’ Mental Fatigue Caused by Sleep Deprivation Using Simultaneous EEG, ECG, and fNIRS Data. Frontiers in Human Neuroscience, 10. https://doi.org/10.3389/fnhum.2016.00219CrossRefGoogle ScholarPubMed
Al-Shargie, F., Kiguchi, M., Badruddin, N., Dass, S. C., Hani, A. F. M., & Tang, T. B. (2016). Mental stress assessment using simultaneous measurement of EEG and fNIRS. Biomedical Optics Express, 7(10), 3882. https://doi.org/10.1364/BOE.7.003882CrossRefGoogle ScholarPubMed
Balters, S., & Steinert, M. (2017). Capturing emotion reactivity through physiology measurement as a foundation for affective engineering in engineering design science and engineering practices. Journal of Intelligent Manufacturing, 28(7), 1585–1607. https://doi.org/10.1007/s10845-015-1145-2CrossRefGoogle Scholar
Blessing, L. T., & Chakrabarti, A. (2009). DRM, a design research methodology. Springer Science & Business Media.CrossRefGoogle Scholar
Cairns, P. E., & Cox, A. L. (2008). Research methods for human-computer interaction. Cambridge University Press.CrossRefGoogle Scholar
Cisler, D., Greenwood, P. M., Roberts, D. M., McKendrick, R., & Baldwin, C. L. (2019). Comparing the Relative Strengths of EEG and Low-Cost Physiological Devices in Modeling Attention Allocation in Semiautonomous Vehicles. Frontiers in Human Neuroscience, 13. https://doi.org/10.3389/fnhum.2019.00109CrossRefGoogle ScholarPubMed
Consolvo, S., Harrison, B., Smith, I., Chen, M. Y., Everitt, K., Froehlich, J., & Landay, J. A. (2007). Conducting In Situ Evaluations for and With Ubiquitous Computing Technologies. International Journal of Human–Computer Interaction, 22(1-2), 103–118. https://doi.org/10.1080/10447310709336957CrossRefGoogle Scholar
Dybvik, H., Erichsen, C. K., & Steinert, M. (2021). DEMONSTRATING THE FEASIBILITY OF MULTIMODAL NEUROIMAGING DATA CAPTURE WITH A WEARABLE ELECTROENCEPHALOGRAPHY + FUNCTIONAL NEAR-INFRARED SPECTROSCOPY (EEG+FNIRS) IN SITU. Proceedings of the Design Society: International Conference on Engineering Design.Google Scholar
Erichsen, C. K., Dybvik, H., & Steinert, M. (2020). Integration of low-cost, dry-comb EEG-electrodes with a standard electrode cap for multimodal signal acquisition during human experiments. DS 101: Proceedings of NordDesign 2020, Lyngby, Denmark, 12th - 14th August 2020, 1–12. https://doi.org/10.35199/NORDDESIGN2020.19CrossRefGoogle Scholar
Frey, J. (2016, May 30). Comparison of a consumer grade EEG amplifier with medical grade equipment in BCI applications. International BCI meeting. https://hal.inria.fr/hal-01278245Google Scholar
Goucher-Lambert, K., Moss, J., & Cagan, J. (2019). A neuroimaging investigation of design ideation with and without inspirational stimuli—Understanding the meaning of near and far stimuli. Design Studies, 60, 1–38. https://doi.org/10.1016/j.destud.2018.07.001CrossRefGoogle Scholar
Hassib, M., Schneegass, S., Eiglsperger, P., Henze, N., Schmidt, A., & Alt, F. (2017). EngageMeter: A System for Implicit Audience Engagement Sensing Using Electroencephalography. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, 5114–5119. https://doi.org/10.1145/3025453.3025669CrossRefGoogle Scholar
Hay, L., Cash, P., & McKilligan, S. (2020). The future of design cognition analysis. Design Science, 6. https://doi.org/10.1017/dsj.2020.20CrossRefGoogle Scholar
Herold, F., Wiegel, P., Scholkmann, F., & Müller, N. G. (2018). Applications of Functional Near-Infrared Spectroscopy (fNIRS) Neuroimaging in Exercise–Cognition Science: A Systematic, Methodology-Focused Review. Journal of Clinical Medicine, 7(12), 466. https://doi.org/10.3390/jcm7120466CrossRefGoogle Scholar
Im, C.-H. (Ed.). (2018). Computational EEG Analysis: Methods and Applications. Springer Singapore. https://doi.org/10.1007/978-981-13-0908-3CrossRefGoogle Scholar
Jacko, J. A. (2012). The human-computer interaction handbook: Fundamentals, evolving technologies, and emerging applications (3rd ed.). CRC Press.CrossRefGoogle Scholar
Lee, S., Shin, Y., Kumar, A. R., Kim, M., & Lee, H. (2019). Dry Electrode-Based Fully Isolated EEG/fNIRS Hybrid Brain-Monitoring System. IEEE Transactions on Biomedical Engineering, 66, 1055–1068. https://doi.org/10.1109/tbme.2018.2866550CrossRefGoogle ScholarPubMed
Liu, Y., Ayaz, H., & Shewokis, P. A. (2017). Multisubject “Learning” for Mental Workload Classification Using Concurrent EEG, fNIRS, and Physiological Measures. Frontiers in Human Neuroscience, 11. https://doi.org/10.3389/fnhum.2017.00389CrossRefGoogle ScholarPubMed
Lukanov, K., Maior, H. A., & Wilson, M. L. (2016). Using fNIRS in Usability Testing: Understanding the Effect of Web Form Layout on Mental Workload. Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, 4011–4016. https://doi.org/10.1145/2858036.2858236CrossRefGoogle Scholar
Malmivuo, J., & Plonsey, R. (1995). Bioelectromagnetism: Principles and Applications of Bioelectric and Biomagnetic Fields. Oxford University Press. https://www.oxfordscholarship.com/view/10.1093/acprof:oso/9780195058239.001.0001/acprof-9780195058239CrossRefGoogle Scholar
Mayseless, N., Hawthorne, G., & Reiss, A. L. (2019). Real-life creative problem solving in teams: FNIRS based hyperscanning study. NeuroImage, 203, 116161. https://doi.org/10.1016/j.neuroimage.2019.116161CrossRefGoogle ScholarPubMed
Nguyen, T. A., & Zeng, Y. (2014). A physiological study of relationship between designer's mental effort and mental stress during conceptual design. Computer-Aided Design, 54, 3–18. https://doi.org/10.1016/j.cad.2013.10.002CrossRefGoogle Scholar
NIRSport. (2015). NIRx Medical Technologies, LLC. https://nirx.net/Google Scholar
Okamoto, M., Dan, H., Shimizu, K., Takeo, K., Amita, T., Oda, I., Konishi, I., Sakamoto, K., Isobe, S., Suzuki, T., Kohyama, K., & Dan, I. (2004). Multimodal assessment of cortical activation during apple peeling by NIRS and fMRI. NeuroImage, 21(4), 1275–1288. https://doi.org/10.1016/j.neuroimage.2003.12.003CrossRefGoogle ScholarPubMed
Oostenveld, R., & Praamstra, P. (2001). The five percent electrode system for high-resolution EEG and ERP measurements. Clinical Neurophysiology, 112(4), 713–719. https://doi.org/10.1016/S1388-2457(00)00527-7CrossRefGoogle ScholarPubMed
OpenBCI Inc. (2019). OpenBCI Inc. https://openbci.com/Google Scholar
Pinti, P., Tachtsidis, I., Hamilton, A., Hirsch, J., Aichelburg, C., Gilbert, S., & Burgess, P. W. (2018). The present and future use of functional near-infrared spectroscopy (fNIRS) for cognitive neuroscience. Annals of the New York Academy of Sciences. http://doi.org/10.1111/nyas.13948CrossRefGoogle Scholar
Piper, S. K., Krueger, A., Koch, S. P., Mehnert, J., Habermehl, C., Steinbrink, J., Obrig, H., & Schmitz, C. H. (2014). A Wearable Multi-Channel fNIRS System for Brain Imaging in Freely Moving Subjects. NeuroImage, 85(01). https://doi.org/10.1016/j.neuroimage.2013.06.062CrossRefGoogle Scholar
Safaie, J., Grebe, R., Moghaddam, H. A., & Wallois, F. (2013). Toward a fully integrated wireless wearable EEG-NIRS bimodal acquisition system. Journal of Neural Engineering, 10(5), 056001. https://doi.org/10.1088/1741-2560/10/5/056001CrossRefGoogle Scholar
Sawan, M., Salam, M. T., Le Lan, J., Kassab, A., Gélinas, S., Vannasing, P., Lesage, F., Lassonde, M., & Nguyen, D. K. (2013). Wireless Recording Systems: From Noninvasive EEG-NIRS to Invasive EEG Devices. IEEE Transactions on Biomedical Circuits and Systems, 7(2), 186–195. https://doi.org/10.1109/TBCAS.2013.2255595CrossRefGoogle ScholarPubMed
Shealy, T., Gero, J., Hu, M., & Milovanovic, J. (2020). Concept generation techniques change patterns of brain activation during engineering design. Design Science, 6. https://doi.org/10.1017/dsj.2020.30CrossRefGoogle Scholar
Shin, J., von Lühmann, A., Blankertz, B., Kim, D.-W., Jeong, J., Hwang, H.-J., & Müller, K.-R. (2017). Open Access Dataset for EEG+NIRS Single-Trial Classification. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 25(10), 1735–1745. https://doi.org/10.1109/TNSRE.2016.2628057CrossRefGoogle ScholarPubMed
Solovey, E. T., Girouard, A., Chauncey, K., Hirshfield, L. M., Sassaroli, A., Zheng, F., Fantini, S., & Jacob, R. J. K. (2009). Using fNIRS brain sensing in realistic HCI settings: Experiments and guidelines. 10.Google Scholar
Steinert, M., & Jablokow, K. (2013). Triangulating front end engineering design activities with physiology data and psychological preferences. 109–118.Google Scholar
Teplan, M. (2002). Fundamentals of EEG Measurement. Measurement Science Review, 2, 11.Google Scholar
The iMotions Platform (8.1). (2020). [Computer software]. iMotions. https://imotions.com/Google Scholar
Ultracortex “Mark IV” EEG Headset. (2019). OpenBCI Inc. https://shop.openbci.com/products/ultracortex-mark-ivGoogle Scholar
Vieira, S., Gero, J. S., Delmoral, J., Gattol, V., Fernandes, C., Parente, M., & Fernandes, A. A. (2020). The neurophysiological activations of mechanical engineers and industrial designers while designing and problem-solving. Design Science, 6. https://doi.org/10.1017/dsj.2020.26CrossRefGoogle Scholar
Vieira, S., Gero, J. S., Delmoral, J., Li, S., Cascini, G., & Fernandes, A. (2020). Brain activity in constrained and open design spaces: An EEG study. Proceedings of the Sixth International Conference on Design Creativity (ICDC 2020), 68–75.CrossRefGoogle Scholar
von Lühmann, A., Herff, C., Heger, D., & Schultz, T. (2015). Toward a Wireless Open Source Instrument: Functional Near-infrared Spectroscopy in Mobile Neuroergonomics and BCI Applications. Frontiers in Human Neuroscience, 9. https://doi.org/10.3389/fnhum.2015.00617CrossRefGoogle Scholar
von Lühmann, A., Wabnitz, H., Sander, T., & Müller, K.-R. (2017). M3BA: A Mobile, Modular, Multimodal Biosignal Acquisition Architecture for Miniaturized EEG-NIRS-Based Hybrid BCI and Monitoring. IEEE Transactions on Biomedical Engineering, 64(6), 1199–1210. https://doi.org/10.1109/TBME.2016.2594127CrossRefGoogle ScholarPubMed
Wulvik, A. S., Dybvik, H., & Steinert, M. (2019). Investigating the relationship between mental state (workload and affect) and physiology in a control room setting (ship bridge simulator). Cognition, Technology & Work. https://doi.org/10.1007/s10111-019-00553-8Google Scholar
Xu, J., Slagle, J. M., Banerjee, A., Bracken, B., & Weinger, M. B. (2019). Use of a Portable Functional Near-Infrared Spectroscopy (fNIRS) System to Examine Team Experience During Crisis Event Management in Clinical Simulations. Frontiers in Human Neuroscience, 13. https://doi.org/10.3389/fnhum.2019.00085CrossRefGoogle ScholarPubMed