Hostname: page-component-76fb5796d-2lccl Total loading time: 0 Render date: 2024-04-28T07:25:18.038Z Has data issue: false hasContentIssue false
  • English
  • Français

Reinforcement learning-based adaptive spiral-diving Manoeuver guidance method for reentry vehicles subject to unknown disturbances

Published online by Cambridge University Press:  19 March 2024

T. Wu Open the ORCID record for T. Wu [Opens in a new window]  and
Z. Wang Open the ORCID record for Z. Wang [Opens in a new window]
T. Wu
Affiliation:
Research Center for Unmanned System Strategy Development, Northwestern Polytechnical University, Xi’an 710072, Shaanxi, China
Z. Wang*
Affiliation:
Research Center for Unmanned System Strategy Development, Northwestern Polytechnical University, Xi’an 710072, Shaanxi, China Unmanned System Research Institute, Northwestern Polytechnical University, Xi’an 710072, Shaanxi, China National Key Laboratory of Aerospace Flight Dynamics, Northwestern Polytechnical University, Xi’an 710072, Shaanxi, China
*
Corresponding author: Z. Wang; Email: nwpu@126.com
Get access Rights & Permissions [Opens in a new window]

Abstract

This paper proposed a reinforcement learning-based adaptive guidance method for a class of spiral-diving manoeuver guidance problems of reentry vehicles subject to unknown disturbances. First, the desired proportional navigation guidance law is designed for the vehicle based on the initial conditions, terminal constraints and the curve involute principle. Then, the first-order multivariable nonlinear guidance command tracking model considering unknown disturbances is established. And the controller design problem caused by the coupling of control variables is overcome by introducing the coordinate transformation technique. Moreover, the actor-critic networks and corresponding adaptive weight update laws are designed to cope with unknown disturbances. With the help of Lyapunov direct method, the stability of the system is proved. Subsequently, the range values of the guidance parameters are analysed. Finally, the validity as well as superiority of the proposed method are verified by numerical simulations.

Keywords

Actor-critic designspiral-diving manoeuveradaptive guidancereentry vehiclesanti-disturbance
Type
Research Article
Information
The Aeronautical Journal , First View , pp. 1 - 17
Check for updates
Copyright
© The Author(s), 2024. Published by Cambridge University Press on behalf of Royal Aeronautical Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

†

Author’s notes

References

Bairstow, S.H. Reentry guidance with extended range capability for low l/d spacecraft, PhD thesis, Massachusetts Institute of Technology, 2006.CrossRefGoogle Scholar
Zhang Mingang, Y.D. A sinking trajectory planning and design method for reentry vehicle under multiple constraints. Missiles Space Veh., 2022, 4, (25–28). doi: 10.7654/j.issn.1004-7182.20220406 Google Scholar
Rahmani-Nejad, A. An anti-hypersonic missiles hybrid optical and enhanced Railgun system. In Counterterrorism, Crime Fighting, Forensics, and Surveillance Technologies V, Vol. 11869, SPIE, Washington State, USA, 2021, pp. 107–116.Google Scholar
Liu, S., Yan, B., Liu, R., Dai, P., Yan, J. and Xin, G. Cooperative guidance law for intercepting a hypersonic target with impact angle constraint. Aeronaut. J., 2022, 126, (1300), pp 1026–1044. doi: 10.1017/aer.2021.117 CrossRefGoogle Scholar
Yu, L.L.X. Weaving maneuver trajectory design for hypersonic glide vehicles, Acta Aeronaut. Astronaut. Sin., 2011, 32, (2174–2181). doi: 10.11-1129/V.20110921.0830.001 Google Scholar
Dwivedi, P., Bhale, P., Bhattacharyya, A. and Padhi, R. Generalized state estimation and model predictive guidance for spiraling and ballistic targets. J. Guid. Control Dyn., 2014, 37, (1), pp 243–264. doi: 10.2514/1.60075 CrossRefGoogle Scholar
He, L., Yan, X. and Tang, S. Spiral-diving trajectory optimization for hypersonic vehicles by second-order cone programming. Aerosp. Sci. Technol., 2019, 95, p. 105427. doi: 10.1016/j.ast.2019.105427 CrossRefGoogle Scholar
Li, G., Zhang, H. and Tang, G. Maneuver characteristics analysis for hypersonic glide vehicles. Aerosp. Sci. Technol., 2015, 43, pp 321–328. doi: 10.1016/j.ast.2015.03.016 CrossRefGoogle Scholar
Rusnak, I and Peled-Eitan, L. Guidance law against spiraling target. J. Guid. Control Dyn., 2016, 39, (7), 1694–1696. doi: 10.2514/1.G001646 CrossRefGoogle Scholar
Yibo, D., Xiaokui, Y., Guangshan, C. and Jiashun, S. Review of control and guidance technology on hypersonic vehicle. Chin. J. Aeronaut., 2022, 35, (7), pp 1–18. doi: 10.1016/j.cja.2021.10.037 Google Scholar
Zhang, W. and Wang, B. A new guidance law for impact angle constraints with time-varying navigation gain. Aeronaut. J., 2022, 126, (1304), pp 1752–1770. doi: 10.1017/aer.2022.16 CrossRefGoogle Scholar
Dou, L. and Dou, J. The design of optimal guidance law with multi-constraints using block pulse functions. Aerosp. Sci. Technol., 2012, 23, (1), pp 201–205. doi: 10.1016/j.ast.2011.02.009 CrossRefGoogle Scholar
Qing, R.M.W. Reentry guidance for hypersonic vehicle based on predictor-corrector method. J. Beijing Univ. Aeronaut. Astronaut., 2013, 39 (1563–1567). doi: 10.13700/j.bh.1001-5965.2013.12.013 Google Scholar
Li, T. and Qian, H. Design of three-dimensional guidance law with impact angle constraints and input saturation. IEEE Access, 2020, 8, pp 211474–211481. doi: 10.1109/ACCESS.2020.3038830 CrossRefGoogle Scholar
Dhananjay, N. and Ghose, D. Accurate time-to-go estimation for proportional navigation guidance. J. Guid. Control Dyn., 2014, 37, (4), pp 1378–1383. doi: 10.2514/1.G000082 CrossRefGoogle Scholar
Mozaffari, M., Safarinejadian, B. and Binazadeh, T. Optimal guidance law based on virtual sliding target. J. Aerosp. Eng., 2017, 30, (3), p 04016097. doi: 10.1061/(ASCE)AS.1943-5525.0000692 CrossRefGoogle Scholar
Raju, P. and Ghose, D. Empirical virtual sliding target guidance law design: an aerodynamic approach. IEEE Trans. Aerosp. Electron. Syst., 2003, 39, (4), pp 1179–1190. doi: 10.1109/TAES.2003.1261120 CrossRefGoogle Scholar
Hu, Q., Han, T. and Xin, M. New impact time and angle guidance strategy via virtual target approach. J. Guid. Control Dyn., 2018, 41, (8), pp 1755–1765. doi: 10.2514/1.G003436 CrossRefGoogle Scholar
He, L. and Yan, X. Adaptive terminal guidance law for spiral-diving maneuver based on virtual sliding targets. J. Guid. Control Dyn., 2018, 41, (7), pp 1591–1601. doi: 10.2514/1.G003424 CrossRefGoogle Scholar
He, L., Yan, X. and Tang, S. Guidance law design for spiral-diving maneuver penetration. Acta Aeronaut. Astronaut. Sin., 2019, 40, (193–207). doi: 10.7527/S1000-6893.2019.22457 Google Scholar
Ning, X., Liu, J., Wang, Z. and Luo, C. Output constrained neural adaptive control for a class of kkvs with non-affine inputs and unmodeled dynamics. Aeronaut. J., 2024, 128, (1319), pp 134–151. doi: 10.1017/aer.2023.44 CrossRefGoogle Scholar
Yin, Z., Wang, B., Xiong, R., Xiang, Z., Liu, L., Fan, H. and Xue, C. Attitude tracking control of hypersonic vehicle based on an improved prescribed performance dynamic surface control. Aeronaut. J. 2023, pp 1–21. doi: 10.1017/aer.2023.79 CrossRefGoogle Scholar
Mao, Q., Dou, L., Yang, Z., Tian, B. and Zong, Q. Fuzzy disturbance observer-based adaptive sliding mode control for reusable launch vehicles with aeroservoelastic characteristic. IEEE Trans. Ind. Inf., 2019, 16, (2), pp 1214–1223. doi: 10.1109/TII.2019.2924731 CrossRefGoogle Scholar
Shen, G., Xia, Y., Zhang, J. and Cui, B. Adaptive super-twisting sliding mode altitude trajectory tracking control for reentry vehicle. ISA Trans., 2023, 132, pp 329–337. doi: 10.1016/j.isatra.2022.06.023 CrossRefGoogle ScholarPubMed
Chai, R., Tsourdos, A., Gao, H., Chai, S. and Xia, Y. Attitude tracking control for reentry vehicles using centralised robust model predictive control. Automatica, 2022, 145, p 110561. doi: 10.1016/j.automatica.2022.110561 CrossRefGoogle Scholar
Cao, L. and Xiao, B. Exponential and resilient control for attitude tracking maneuvering of spacecraft with actuator uncertainties. IEEE/ASME Trans. Mechatron., 2019, 24, (6), pp 2531–2540. doi: 10.1109/TMECH.2019.2928703 CrossRefGoogle Scholar
Xiang, K., Yanli, D., Peng, Z. and Haibing, L. Adaptive backstepping control for hypersonic vehicles with actuator amplitude and rate saturation. Trans. Nanjing Univ. Aeronaut. Astronaut., 2019, 36, (2), pp 242–252. doi: 10.16356/j.1005-1120.2019.02.007 Google Scholar
Cheng, L., Wang, Z. and Gong, S. Adaptive control of hypersonic vehicles with unknown dynamics based on dual network architecture. Acta Astronaut., 2022, 193, pp 197–208. doi: 10.1016/j.actaastro.2021.12.043 CrossRefGoogle Scholar
Cheng, L., Wang, Z., Jiang, F. and Li, J. Fast generation of optimal asteroid landing trajectories using deep neural networks. IEEE Trans. Aerosp. Electron. Syst., 2019, 56, (4), pp 2642–2655. doi: 10.1109/TAES.2019.2952700 CrossRefGoogle Scholar
Zhang, X., Chen, K., Fu, W. and Huang, H. Neural network-based stochastic adaptive attitude control for generic hypersonic vehicles with full state constraints. Neurocomputing, 2019, 351, pp 228–239. doi: 10.1016/j.neucom.2019.04.014 CrossRefGoogle Scholar
Ouyang, Y., Dong, L., Wei, Y. and Sun, C. Neural network based tracking control for an elastic joint robot with input constraint via actor-critic design. Neurocomputing, 2020, 409, pp 286–295. doi: 10.1016/j.neucom.2020.05.067 CrossRefGoogle Scholar
Shi, L., Wang, X. and Cheng, Y. Safe reinforcement learning-based robust approximate optimal control for hypersonic flight vehicles. IEEE Trans. Veh. Technol., 2023. doi: 10.1109/TVT.2023.3264243 CrossRefGoogle Scholar
Wang, Z. and Liu, J. Reinforcement learning based-adaptive tracking control for a class of semi-markov non-lipschitz uncertain system with unmatched disturbances. Inf. Sci. 2023, 626, pp 407–427. doi: 10.1016/j.ins.2023.01.043 CrossRefGoogle Scholar
He, W. and Dong, Y. Adaptive fuzzy neural network control for a constrained robot using impedance learning. IEEE Trans. Neural Netwk. Learn. Syst., 2017, 29, (4), pp 1174–1186. doi: 10.1109/TNNLS.2017.2665581 CrossRefGoogle ScholarPubMed
Xing-Kai, H. Young type inequalities for matrices. J. East China Normal Univ. (Nat. Sci.), 2012, 2012, (4), p 12. doi: 10.3969/j.issn.1000-5641.2012.04.002 Google Scholar