Hostname: page-component-848d4c4894-sjtt6 Total loading time: 0 Render date: 2024-06-19T12:06:06.412Z Has data issue: false hasContentIssue false
  • English
  • Français

Safe and socially compliant robot navigation in crowds with fast-moving pedestrians via deep reinforcement learning

Published online by Cambridge University Press:  26 February 2024

Zhen Feng Open the ORCID record for Zhen Feng [Opens in a new window] ,
Bingxin Xue ,
Chaoqun Wang  and
Fengyu Zhou
Zhen Feng
Affiliation:
School of Control Science and Engineering, Shandong University, Jinan, Shandong, China
Bingxin Xue
Affiliation:
School of Control Science and Engineering, Shandong University, Jinan, Shandong, China
Chaoqun Wang
Affiliation:
School of Control Science and Engineering, Shandong University, Jinan, Shandong, China
Fengyu Zhou*
Affiliation:
School of Control Science and Engineering, Shandong University, Jinan, Shandong, China
*
Corresponding author: Fengyu Zhou; Email: zhoufengyu@sdu.edu.cn
Get access Rights & Permissions [Opens in a new window]

Abstract

Safe and socially compliant navigation in a crowded environment is essential for social robots. Numerous research efforts have shown the advantages of deep reinforcement learning techniques in training efficient policies, while most of them ignore fast-moving pedestrians in the crowd. In this paper, we present a novel design of safety measure, named Risk-Area, considering collision theory and motion characteristics of different robots and humans. The geometry of Risk-Area is formed based on the real-time relative positions and velocities of the agents in the environment. Our approach perceives risk in the environment and encourages the robot to take safe and socially compliant navigation behaviors. The proposed method is verified with three existing well-known deep reinforcement learning models in densely populated environments. Experiment results demonstrate that our approach combined with the reinforcement learning techniques can efficiently perceive risk in the environment and navigate the robot with high safety in the crowds with fast-moving pedestrians.

Keywords

Navigationmobile robotshuman safety and comfortsocial roboticscollision theorydeep reinforcement learning
Type
Research Article
Information
Robotica , Volume 42 , Issue 4 , April 2024 , pp. 1212 - 1230
Copyright
© The Author(s), 2024. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Kruse, T., Pandey, A. K., Alami, R. and Kirsch, A., “Human-aware robot navigation: A survey,” Robot Auton Syst 61(12), 1726–1743 (2013).CrossRefGoogle Scholar
Charalampous, K., Kostavelis, I. and Gasteratos, A., “Recent trends in social aware robot navigation: A survey,” Robot Auton Syst 93, 85–104 (2017).CrossRefGoogle Scholar
Malviya, V., Reddy, A. K. and Kala, R., “Autonomous social robot navigation using a behavioral finite state social machine,” Robotica 38(12), 2266–2289 (2020).CrossRefGoogle Scholar
Yin, L. and Yin, Y., “An Improved Potential Field Method for Mobile Robot Path Planning in Dynamic Environments,” In: 2008 7th World Congress On Intelligent Control and Automation (WCICA), Chongqing, China (IEEE, 2008) pp. 4847–4852.Google Scholar
Fan, T., Cheng, X., Pan, J., Long, P., Liu, W., Yang, R. and Manocha, D., “Getting robots unfrozen and unlost in dense pedestrian crowds,” IEEE Robot Autom Lett 4(2), 1178–1185 (2019).CrossRefGoogle Scholar
Trautman, P. and Krause, A., “Unfreezing the Robot: Navigation in Dense, Interacting Crowds,” In: 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Taipei, Taiwan (IEEE, 2010) pp. 797–803.Google Scholar
Sathyamoorthy, A. J., Patel, U., Guan, T. and Manocha, D., “Frozone: Freezing-free, pedestrian-friendly navigation in human crowds,” IEEE Robot Autom Lett 5(3), 4352–4359 (2020).CrossRefGoogle Scholar
Bachiller, P., Rodriguez-Criado, D., Jorvekar, R. R., Bustos, P., Faria, D. R. and Manso, L. J., “A graph neural network to model disruption in human-aware robot navigation,” Multimed Tools Appl 81(3), 3277–3295 (2022).CrossRefGoogle Scholar
Charalampous, K., Kostavelis, I. and Gasteratos, A., “Robot navigation in large-scale social maps: An action recognition approach,” Expert Syst Appl 66, 261–273 (2016).CrossRefGoogle Scholar
Li, K., Xu, Y., Wang, J. and Meng, M. Q.-H., “SARL: Deep Reinforcement Learning Based Human-Aware Navigation for Mobile Robot in Indoor Environments,” In: 2019 IEEE International Conference on Robotics and Biomimetics (ROBIO), Dali, China (IEEE, 2019) pp. 688–694.CrossRefGoogle Scholar
Truong, X.-T. and Ngo, T.-D., “To approach humans?: A unified framework for approaching pose prediction and socially aware robot navigation,” IEEE Trans Cogn Develop Syst 10(3), 557–572 (2018).CrossRefGoogle Scholar
Pfeiffer, M., Schwesinger, U., Sommer, H., Galceran, E. and Siegwart, R., “Predicting Actions to Act Predictably: Cooperative Partial Motion Planning with Maximum Entropy Models,” In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, Korea (South) (IEEE, 2016) pp. 2096–2101.CrossRefGoogle Scholar
Bennewitz, M., Burgard, W., Cielniak, G. and Thrun, S., “Learning motion patterns of people for compliant robot motion,” Int J Robot Res 24(1), 31–48 (2005).Google Scholar
Agarwal, P., Kumar, S., Ryde, J., J. Corso, V. Krovi, N. Ahmed, J. Schoenberg, M. Campbell, M. Bloesch, M. Hutter, M. Hoepflinger, S. Leutenegger, C. Gehring, C. David Remy, R. Siegwart, J. Brookshire, S. Teller, M. Bryson, M. Johnson-Roberson, … P. R. Giordano, Feature-based prediction of trajectories for socially compliant navigation. (2013). (MIT Press, USA).Google Scholar
Aoude, G. S., Luders, B. D., Joseph, J. M., Roy, N. and How, J. P., “Probabilistically safe motion planning to avoid dynamic obstacles with uncertain motion patterns,” Auton Robots 35(1), 51–76 (2013).CrossRefGoogle Scholar
Zhou, Z., Zhu, P., Zeng, Z., Xiao, J., Lu, H. and Zhou, Z., “Robot navigation in a crowd by integrating deep reinforcement learning and online planning,” Appl Intell 52(13), 15600–15616 (2022).CrossRefGoogle Scholar
Sun, L., Zhai, J. and Qin, W., “Crowd navigation in an unknown and dynamic environment based on deep reinforcement learning,” IEEE Access 7, 109544–109554 (2019).CrossRefGoogle Scholar
Hu, Z., Zhao, Y., Zhang, S., Zhou, L. and Liu, J., “Crowd-comfort robot navigation among dynamic environment based on social-stressed deep reinforcement learning,” Int J Soc Robot 14(4), 913–929 (2022).Google Scholar
Samsani, S. S. and Muhammad, M. S., “Socially compliant robot navigation in crowded environment by human behavior resemblance using deep reinforcement learning,” IEEE Robot Autom Lett 6(3), 5223–5230 (2021).CrossRefGoogle Scholar
Helbing, D. and Molnar, P., “Social force model for pedestrian dynamics,” Phys Rev E 51(5), 4282–4286 (1995).CrossRefGoogle ScholarPubMed
Ferrer, G., Garrell, A. and Sanfeliu, A., “Robot Companion: A Social-Force Based Approach with Human Awareness-Navigation in Crowded Environments,” In: 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Tokyo, Japan (IEEE, 2013) pp. 1688–1694.CrossRefGoogle Scholar
Yang, C., Zhang, T., Chen, L.-P. and Fu, L.-C., “Socially-Aware Navigation of Omnidirectional Mobile Robot with Extended Social Force Model in Multi-Human Environment,” In: 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC), Bari, Italy (IEEE, 2019) pp. 1963–1968.Google Scholar
van den Berg, J., Lin, M. and Manocha, D., “Reciprocal Velocity Obstacles for Real-Time Multi-Agent Navigation,” In: IEEE International Conference on Robotics and Automation (ICRA) 2008, Pasadena, CA (IEEE, 2008) pp. 1928–1935.Google Scholar
van den Berg, J., Guy, S. J., Lin, M. and Manocha, D., “Reciprocal n-body collision avoidance,” In: Proceedings of the Robotics Research, Berlin, Heidelberg (Springer, 2011) pp. 3–19.Google Scholar
Trautman, P., Ma, J., Murray, R. M. and Krause, A., “Robot Navigation in Dense Human Crowds: The Case for Cooperation,” In: IEEE International Conference on Robotics and Automation (ICRA) 2013, Karlsruhe, Germany (IEEE, 2013) pp. 2153–2160.Google Scholar
Tai, L., Zhang, J., Liu, M. and Burgard, W., “Socially Compliant Navigation Through Raw Depth Inputs with Generative Adversarial Imitation Learning,” In: IEEE International Conference on Robotics and Automation (ICRA) 2018, Brisbane, Australia (IEEE, 2018) pp. 1111–1117.Google Scholar
Wu, Q., Gong, X., Xu, K., Manocha, D., Dong, J. and Wang, J., “Towards target-driven visual navigation in indoor scenes via generative imitation learning,” IEEE Robot Autom Lett 6(1), 175–182 (2021).Google Scholar
Qin, L., Huang, Z., Zhang, C., Guo, H., Ang, M. and Rus, D., “Deep Imitation Learning for Autonomous Navigation in Dynamic Pedestrian Environments,” In: 2021 IEEE International Conference on Robotics and Automation (ICRA), Xi’an, China (IEEE, 2021) pp. 4108–4115.CrossRefGoogle Scholar
Kretzschmar, H., Spies, M., Sprunk, C. and Burgard, W., “Socially compliant mobile robot navigation via inverse reinforcement learning,” Int J Robot Res 35(11), 1289–1307 (2016).CrossRefGoogle Scholar
Kollmitz, M., Koller, T., Boedecker, J. and Burgard, W., “Learning Human-Aware Robot Navigation from Physical Interaction via Inverse Reinforcement Learning,” In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, NV, USA (IEEE, 2020) pp. 11025–11031.Google Scholar
Konar, A., Baghi, B. H. and Dudek, G., “Learning goal conditioned socially compliant navigation from demonstration using risk-based features,” IEEE Robot Autom Lett 6(2), 651–658 (2021).CrossRefGoogle Scholar
Long, P., Fan, T., Liao, X., Liu, W., Zhang, H. and Pan, J., “Towards Optimally Decentralized Multi-Robot Collision Avoidance via Deep Reinforcement Learning,” In: 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, QLD, Australia (IEEE, 2018) pp. 6252–6259.CrossRefGoogle Scholar
Chen, Y. F., Liu, M., Everett, M. and How, J. P., “Decentralized Non-Communicating Multiagent Collision Avoidance with Deep Reinforcement Learning,” In: 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore (IEEE, 2017) pp. 285–292.CrossRefGoogle Scholar
Chen, Y. F., Everett, M., Liu, M. and How, J. P., “Socially Aware Motion Planning with Deep Reinforcement Learning,” In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada (IEEE, 2017) pp. 1343–1350.Google Scholar
Everett, M., Chen, Y. F. and How, J. P., “Motion Planning Among Dynamic, Decision-Making Agents with Deep Reinforcement Learning,” In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain (IEEE, 2018) pp. 3052–3059.CrossRefGoogle Scholar
Chen, C., Liu, Y., Kreiss, S. and Alahi, A., “Crowd-Robot Interaction: Crowd-Aware Robot Navigation with Attention-Based Deep Reinforcement Learning“ In: IEEE International Conference on Robotics and Automation (ICRA) 2019, Montreal, QC, Canada (IEEE, 2019) pp. 6015–6022.CrossRefGoogle Scholar
Chen, Y., Liu, C., Shi, B. E. and Liu, M., “Robot navigation in crowds by graph convolutional networks with attention learned from human gaze,” IEEE Robot Autom Lett 5(2), 2754–2761 (2020).CrossRefGoogle Scholar
Bohannon, R. W. and Andrews, A. W., “Normal walking speed: A descriptive meta-analysis,” Physiotherapy 97(3), 182–189 (2011).CrossRefGoogle ScholarPubMed
Butler, J. T. and Agah, A., “Psychological effects of behavior patterns of a mobile personal robot,” Auton Robots 10(2), 185–202 (2001).Google Scholar
Hoyt, D. F. and Taylor, C. R., “Gait and the energetics of locomotion in horses,” Nature 292(5820), 239–240 (1981).Google Scholar
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., Petersen, S., Beattie, C., Sadik, A., Antonoglou, I., King, H., Kumaran, D., Wierstra, D., Legg, S. and Hassabis, D., “Human-level control through deep reinforcement learning,” Nature 518(7540), 529–533 (2015).Google ScholarPubMed
Bera, A. and Manocha, D., “Realtime Multilevel Crowd Tracking Using Reciprocal Velocity Obstacles,” In: 2014 22nd International Conference on Pattern Recognition (ICPR), Stockholm, Sweden (IEEE, 2014) pp. 4164–4169.Google Scholar
Sutton, R. S. and Barto, A. G..Reinforcement learning: An introduction. (2018). (MIT Press, USA).Google Scholar
Bai, T., Fan, Z., Liu, M., Zhang, S. and Zheng, R., “Multiple Waypoint Path Planning for a Home Mobile Robot,” In: 2018 Ninth International Conference on Intelligent Control and Information Processing (ICICIP), Wanzhou, China, (IEEE, 2018) pp. 53–58.CrossRefGoogle Scholar
Watts, C. M., Lancaster, P., Pedross-Engel, A., Smith, J. R. and Reynolds, M. S.. 2D and 3D Millimeter-Wave Synthetic Aperture Radar Imaging on a PR2 Platform. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, Korea (South) (IEEE, 2016) pp. 4304–4310.CrossRefGoogle Scholar
Brach, R. M., Mechanical Impact Dynamics: Rigid Body Collisions (Wiley, New York, 1991).Google Scholar
Brach, R. M., “Rigid body collisions,” J Appl Mech 56(1), 133–138 (1989).CrossRefGoogle Scholar
Minetti, A. E., “Chapter 5 - The three modes of Terrestrial locomotion,” In: Biomechanics and Biology of Movement (Human Kinetics, Champaign, 2000) pp. 67–78.Google Scholar