Hostname: page-component-84b7d79bbc-g5fl4 Total loading time: 0 Render date: 2024-07-25T10:57:06.472Z Has data issue: false hasContentIssue false
  • English
  • Français

DESIGN OF A TELEOPERATION USER INTERFACE FOR SHARED CONTROL OF HIGHLY AUTOMATED AGRICULTURAL MACHINESS

Published online by Cambridge University Press:  19 June 2023

Sebastian Lorenz
Sebastian Lorenz*
Affiliation:
Technische Universität Dresden
*
Lorenz, Sebastian, Technische Universität Dresden, Germany, sebastian.lorenz3@tu-dresden.de

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

This paper presents a focused examination of critical performance and design issues for the introduction of highly automated tractors and their user interfaces in agriculture. An industry that as of today mainly uses direct-controlled machines that at least to some extent have partly automated functionalities. Issues include out-of-the-loop unfamiliarity, interface complexity, automation transparency, and changing information modalities in teleoperation scenarios for former cabin-based operated machines. Selected evidence and accompanying concepts and findings from literature are put in context to each issue, informing a systematic design process that utilizes the frameworks of knowledge engineering and ecological interface design. The resulting user interface prototype is built upon the identified requirements in analysis and collected design guidelines, stemming from various research areas. The documentation of the consideration of these in context with additional requirements, such as complexity reduction, information interactivity, and users' existing experiences is meant to provide insights into the often opaque and art-like design space.

Keywords

Design engineeringKnowledge managementUser centred designDesign for interfaces
Type
Article
Information
Proceedings of the Design Society , Volume 3: ICED23 , July 2023 , pp. 1277 - 1286
Creative Commons
Creative Common License - CCCreative Common License - BYCreative Common License - NCCreative Common License - ND
This is an Open Access article, distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives licence (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is unaltered and is properly cited. The written permission of Cambridge University Press must be obtained for commercial re-use or in order to create a derivative work.
Copyright
The Author(s), 2023. Published by Cambridge University Press

References

Ackerman, K.A., Talleur, D.A., Carbonari, R.S., Xargay, E., Seefeldt, B.D., Kirlik, A., Hovakimyan, N. and Trujillo, A.C. (2017), “Automation Situation Awareness Display for a Flight Envelope Protection System”, Journal of Guidance, Control, and Dynamics, Vol. 40 No. 4, pp. 964–980.CrossRefGoogle Scholar
Arnold, D., Arntz, M., Gregory, T., Steffes, S. and Zierahn, U. (2016), “Herausforderungen der Digitalisierung für die Zukunft der Arbeitswelt”, ZEW policy brief, Vol. 16 No. 8.Google Scholar
Averjanova, O., Ricci, F. and Nguyen, Q.N. (2008), “Map-Based Interaction with a Conversational Mobile Recommender System”, in 2008 The Second International Conference on Mobile Ubiquitous Computing, Systems, Services and Technologies, IEEE.CrossRefGoogle Scholar
Bandura, A. (2006), “Guide for constructing self-efficacy scales”, Self-efficacy beliefs of adolescents, Vol. 5 No. 1, pp. 307–337.Google Scholar
Chen, J.Y.C., Barnes, M.J. and Harper-Sciarini, M. (2011), “Supervisory Control of Multiple Robots: Human-Performance Issues and User-Interface Design”, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews ), Vol. 41 No. 4, pp. 435–454.Google Scholar
Debernard, S., Chauvin, C., Pokam, R. and Langlois, S. (2016), “Designing Human-Machine Interface for Autonomous Vehicles”, IFAC-PapersOnLine, Vol. 49 No. 19, pp. 609–614.CrossRefGoogle Scholar
DeJong, B.P., Colgate, J.E. and Peshkin, M.A. (2004), “Improving teleoperation: reducing mental rotations and translations”, in IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004, IEEE.CrossRefGoogle Scholar
Doi, T. (2019), “Mental Model Formation in Users with High and Low Comprehension of a Graphical User Interface”, Journal of Human Ergology, Vol. 48 No. 1, pp. 9–24.Google Scholar
Endsley, M.R., Bolte, B. and Jones, D.G. (2003), Designing for Situation Awareness, CRC press.CrossRefGoogle Scholar
Endsley, M.R. and Kiris, E.O. (1995), “The Out-of-the-Loop Performance Problem and Level of Control in Automation”, Human Factors: The Journal of the Human Factors and Ergonomics Society, Vol. 37 No. 2, pp. 381–394.CrossRefGoogle Scholar
Fong, T., Thorpe, C., & Baur, C (2001), “Advanced Interfaces for Vehicle Teleoperation: Collaborative Control, Sensor Fusion Displays, and Remote Driving Tools”, Autonomous Robots, Vol. 11 No. 1, pp. 77–85.CrossRefGoogle Scholar
Ricci, Francesco, Nguyen, Quang Nhat and Averjanova, Olga, “Exploiting a Map-Based Interface in Conversational Recommender Systems for Mobile Travelers”, in Tourism Informatics: Visual Travel Recommender Systems, Social Communities, and User Interface Design, IGI Global, pp. 73–93.CrossRefGoogle Scholar
Hoff, K.A. and Bashir, M. (2015), “Trust in automation: integrating empirical evidence on factors that influence trust”, Human Factors: The Journal of the Human Factors and Ergonomics Society, Vol. 57 No. 3, pp. 407–434.CrossRefGoogle ScholarPubMed
Holsopple, J., Sudit, M., Nusinov, M., Liu, D., Du, H. and Yang, S. (2010), “Enhancing situation awareness via automated situation assessment”, IEEE Communications Magazine, Vol. 48 No. 3, pp. 146–152.CrossRefGoogle Scholar
Kantorowitz, E. and Sudarsky, O. (1989), “The adaptable user interface”, Communications of the ACM, Vol. 32 No. 11, pp. 1352–1358.CrossRefGoogle Scholar
Kessler, F., Lorenz, S., Miesen, F., Miesner, J., Pelzer, F., Satkowski, M., Klose, A. and Urbas, L. (2022), “Conducive Design as an Iterative Process for Engineering CPPS”, in Human Aspects of Advanced Manufacturing, AHFE International.CrossRefGoogle Scholar
Kondo, B. and Collins, C. (2014), “DimpVis: Exploring Time-varying Information Visualizations by Direct Manipulation”, IEEE Transactions on Visualization and Computer Graphics, Vol. 20 No. 12, pp. 2003–2012.CrossRefGoogle ScholarPubMed
Kühme, T. (1993), “A user-centered approach to adaptive interfaces”, in Proceedings of the 1st international conference on Intelligent user interfaces - IUI '93, New York, New York, USA, ACM Press, New York, New York, USA.CrossRefGoogle Scholar
Lim, B.Y. and Dey, A.K. (2009), “Assessing demand for intelligibility in context-aware applications”, in Proceedings of the 11th international conference on Ubiquitous computing, New York, NY, USA, ACM, New York, NY, USA.CrossRefGoogle Scholar
Masoodian, M. and Luz, S. (06062022), “Map-based Interfaces and Interactions”, in Bottoni, P. and Panizzi, E. (Eds.), Proceedings of the 2022 International Conference on Advanced Visual Interfaces, 06 06 2022 10 06 2022, Frascati, Rome Italy, ACM, New York, NY, USA, pp. 1–4.Google Scholar
Meier, R., Fong, T., Thorpe, C. and Baur, C., “A Sensor Fusion based user interface for vehicle teleoperation”, in Field and Service Robotics.Google Scholar
Merritt, S.M. (2011), “Affective processes in human-automation interactions”, Human Factors: The Journal of the Human Factors and Ergonomics Society, Vol. 53 No. 4, pp. 356–370.CrossRefGoogle Scholar
Moray, N. (2003), “Monitoring, complacency, scepticism and eutactic behaviour”, International Journal of Industrial Ergonomics, Vol. 31 No. 3, pp. 175–178.CrossRefGoogle Scholar
Muslim, H. and Itoh, M. (2019), “A theoretical framework for designing human-centered automotive automation systems”, Cognition, Technology & Work, Vol. 21 No. 4, pp. 685–697.CrossRefGoogle Scholar
Navarro, J., Heuveline, L., Avril, E. and Cegarra, J. (2018), “Influence of human-machine interactions and task demand on automation selection and use”, Ergonomics, Vol. 61 No. 12, pp. 1601–1612.CrossRefGoogle Scholar
Nuamah, J. and Seong, Y. (2017 - 2017), “Human machine interface in the Internet of Things (IoT)”, in 2017 12th System of Systems Engineering Conference (SoSE), 18.06.2017 - 21.06.2017, Waikoloa, HI, USA, IEEE, pp. 1–6.Google Scholar
Onnasch, L. and Hösterey, S. (2019), “Stages of Decision Automation: Impact on Operators’ Role, Awareness and Monitoring”, Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Vol. 63 No. 1, pp. 282–286.CrossRefGoogle Scholar
Opiyo, S., Zhou, J., Mwangi, E., Kai, W. and Sunusi, I. (2021), “A Review on Teleoperation of Mobile Ground Robots: Architecture and Situation Awareness”, International Journal of Control, Automation and Systems, Vol. 19 No. 3, pp. 1384–1407.CrossRefGoogle Scholar
Paelke, V., Nebe, K., Geiger, C., Klompmaker, F., & Fischer, H (2012), Designing multi-modal map-based interfaces for disaster management.Google Scholar
Panteli, M., Kirschen, D.S., Crossley, P.A. and Sobajic, D.J. (2013), “Enhancing situation awareness in power system control centers”, in 2013 IEEE International Multi-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support (CogSIMA), IEEE.Google Scholar
Parasuraman, R. and Manzey, D.H. (2010), “Complacency and bias in human use of automation: an attentional integration”, Human Factors, Vol. 52 No. 3, pp. 381–410.CrossRefGoogle ScholarPubMed
Parasuraman, R., Sheridan, T.B. and Wickens, C.D. (2008), “Situation Awareness, Mental Workload, and Trust in Automation: Viable, Empirically Supported Cognitive Engineering Constructs”, Journal of Cognitive Engineering and Decision Making, Vol. 2 No. 2, pp. 140–160.CrossRefGoogle Scholar
Pedersen, S.M., Fountas, S. and Blackmore, S. (2018), “Agricultural Robots - Applications and Economic Perspectives”, in Takahashi, Yoshihiko (Ed.), Service Robot Applications.Google Scholar
Picking, R., Grout, V., McGinn, J., Crisp, J. and Grout, H. (2010), “Simplicity, Consistency, Universality, Flexibility and Familiarity”, International Journal of Ambient Computing and Intelligence, Vol. 2 No. 3, pp. 40–49.CrossRefGoogle Scholar
Plaisant, C., Grosjean, J. and Bederson, B.B. (2002), “SpaceTree: supporting exploration in large node link tree, design evolution and empirical evaluation”, in IEEE Symposium on Information Visualization, IEEE Comput. Soc.CrossRefGoogle Scholar
Pokam, R., Debernard, S., Chauvin, C. and Langlois, S. (2019), “Principles of transparency for autonomous vehicles: first results of an experiment with an augmented reality human–machine interface”, Cognition, Technology & Work, Vol. 21 No. 4, pp. 643–656.CrossRefGoogle Scholar
Rapp, D.N. (2007), “Mental Models: Theoretical Issues for Visualizations in Science Education”, in Gilbert, J.K. (Ed.), Visualization in science education, Models and modeling in science education, Springer, Dordrecht, pp. 43–60.Google Scholar
Reina, G., Milella, A., Rouveure, R., Nielsen, M., Worst, R. and Blas, M.R. (2016), “Ambient awareness for agricultural robotic vehicles”, Biosystems Engineering, Vol. 146, pp. 114–132.CrossRefGoogle Scholar
Robertson, G., Czerwinski, M., Fisher, D. and Lee, B. (2009), “Selected Human Factors Issues in Information Visualization”, Reviews of Human Factors and Ergonomics, Vol. 5 No. 1, pp. 41–81.CrossRefGoogle Scholar
Sandouka, K. (2019), Interactive visualizations: A literature review.Google Scholar
Sarter, N.B. (2002), “Multimodal information presentation in support of human-automation communication and coordination”.CrossRefGoogle Scholar
Sarter, N.B. (2006), “Multimodal information presentation: Design guidance and research challenges”, International Journal of Industrial Ergonomics, Vol. 36 No. 5, pp. 439–445.CrossRefGoogle Scholar
Schwarzer, R., & Jerusalem, M. (2002), “Das Konzept der Selbstwirksamkeit”, in Jerusalem, M. and Hopf, D. (Eds.), Selbstwirksamkeit und Motivationsprozesse in Bildungsinstitutionen, Zeitschrift für Pädagogik Beiheft, Unveränd. Nachdr. der letzten Aufl., Beltz, Weinheim, pp. 28–53.Google Scholar
Stuerzlinger, W., Chapuis, O., Phillips, D. and Roussel, N. (2006), “User interface façades”, in Proceedings of the 19th annual ACM symposium on User interface software and technology, New York, NY, USA, ACM, New York, NY, USA.CrossRefGoogle Scholar
Top, F., Pütz, S. and Fottner, J. (2021), “Human-Centered HMI for Crane Teleoperation: Intuitive Concepts Based on Mental Models, Compatibility and Mental Workload”, in 2021 International Conference on Human-Computer Interaction, Springer, Cham, pp. 438–456.Google Scholar
Tversky, B., Morrison, J. and Betrancourt, M. (2002), “Animation: can it facilitate?”, International Journal of Human-Computer Studies, Vol. 57 No. 4, pp. 247–262.CrossRefGoogle Scholar
Vasconez, J.P., Kantor, G.A. and Auat Cheein, F.A. (2019), “Human–robot interaction in agriculture: A survey and current challenges”, Biosystems Engineering, Vol. 179, pp. 35–48.CrossRefGoogle Scholar
Wickens, C. (2018), “Automation Stages & Levels, 20 Years After”, Journal of Cognitive Engineering and Decision Making, Vol. 12 No. 1, pp. 35–41.CrossRefGoogle Scholar
Wickens, C.D. (2002), “Situation Awareness and Workload in Aviation”, Current Directions in Psychological Science, Vol. 11 No. 4, pp. 128–133.CrossRefGoogle Scholar
Zohrevandi, E., Westin, C.A.L., Lundberg, J. and Ynnerman, A. (2022), “Design and Evaluation Study of Visual Analytics Decision Support Tools in Air Traffic Control”, Computer Graphics Forum, Vol. 41 No. 1, pp. 230–242.CrossRefGoogle Scholar