Hostname: page-component-77c89778f8-gq7q9 Total loading time: 0 Render date: 2024-07-19T21:49:20.217Z Has data issue: false hasContentIssue false


Published online by Cambridge University Press:  27 July 2021

Felician Campean*
University of Bradford;
Sohag Kabir
University of Bradford;
Cuong Dao
University of Bradford;
Qichun Zhang
University of Bradford;
Claudia Eckert
The Open University
Campean, Felician, University of Bradford School of Engineering United Kingdom,


Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

Applications of autonomous systems are becoming increasingly common across the field of engineered systems from cars, drones, manufacturing systems and medical devices, addressing prevailing societal changes, and, increasingly, consumer demand. Autonomous systems are expected to self-manage and self-certify against risks affecting the mission, safety and asset integrity. While significant progress has been achieved in relation to the modelling of safety and safety assurance of autonomous systems, no similar approach is available for resilience that integrates coherently across the cyber and physical parts. This paper presents a comprehensive discussion of resilience in the context of robotic autonomous systems, covering both resilience by design and resilience by reaction, and proposes a conceptual model of a system of learning for resilience assurance in a continuous product development framework. The resilience assurance model is proposed as a composable digital artefact, underpinned by a rigorous model-based resilience analysis at the system design stage, and dynamically monitored and continuously updated at run time in the system operation stage, with machine learning based knowledge extraction and validation.

Creative Commons
Creative Common License - CCCreative Common License - BYCreative Common License - NCCreative Common License - ND
This is an Open Access article, distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives licence (, which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is unaltered and is properly cited. The written permission of Cambridge University Press must be obtained for commercial re-use or in order to create a derivative work.
The Author(s), 2021. Published by Cambridge University Press


Avizienis, A., Laprie, J., Randell, B., Landwehr, C., (2004). Basic concepts and taxonomy of dependable and secure computing. IEEE Trans. Dependable Secure Comput. 1, 1133. DOI: 10.1109/TDSC.2004.2Google Scholar
Bagchi, S. et al. (2020). Vision Paper: Grand Challenges in Resilience: Autonomous System Resilience through Design and Runtime Measures, IEEE Open Jrnl Comp Soc, 1:155172, DOI: 10.1109/OJCS.2020.3006807CrossRefGoogle Scholar
Baresi, L., Pasquale, L., & Spoletini, P. (2010). Fuzzy goals for requirements-driven adaptation. In 18th IEEE International Requirements Engineering Conference, pp. 125134, IEEE. DOI: 10.1109/RE.2010.25Google Scholar
Becker, B., Beyer, D., Giese, H., Klein, F., & Schilling, D. (2006). Symbolic invariant verification for systems with dynamic structural adaptation. In Proc 28th Int Conf Soft Eng, DOI: 10.1145/1134285.1134297Google Scholar
Calinescu, R., Weyns, D., Gerasimou, S., Iftikhar, M. U., Habli, I., & Kelly, T. (2018). Engineering trustworthy self-adaptive software with dynamic assurance cases. IEEE Trans Soft Eng, 44(11), 10391069. DOI: 10.1109/TSE.2017.2738640CrossRefGoogle Scholar
Cámara, J., de Lemos, R., Laranjeiro, N., Ventura, R., & Vieira, M. (2015). Robustness-driven resilience evaluation of self-adaptive software systems. IEEE Trans Dep & Secure Computing, 14(1), 5064. DOI: 10.1109/TDSC.2015.2429128Google Scholar
Campean, F, Delaux, D, Sharma, S., Bridges, J., (2020) Reliability Research Roadmapping Workshop: Implications for Engineering Design, Proc Des Soc, DOI: 10.1017/dsd.2020.337Google Scholar
Cheng, B.H., Eder, K.I., Gogolla, M., Grunske, L., Litoiu, M., Müller, H.A., Pelliccione, P., Perini, A., Qureshi, N.A., Rumpe, B. and Schneider, D. (2014). Using models at runtime to address assurance for self-adaptive systems. In Models@ run. time, pp. 101136. DOI: 10.1007/978-3-319-08915-7_4Google Scholar
Connelly, E. B., Allen, C. R., Hatfield, K., Palma-Oliveira, J. M., Woods, D. D. and Linkov, I. (2017) Features of resilience, Environ. Syst. Decis., 37:1: 4650, DOI:10.1007/s10669-017-9634-9.CrossRefGoogle Scholar
Fisher, M., Mascardi, V., Rozier, K.Y., Schlingloff, B.H., Winikoff, M. and Yorke-Smith, N., (2021). Towards a framework for certification of reliable autonomous systems. Autonomous Agents and Multi-Agent Systems, 35(1), pp.165. DOI: 10.1007/s10458-020-09487-2Google Scholar
Fredericks, E. M., Ramirez, A. J., & Cheng, B. H. (2013). Validating code-level behavior of dynamic adaptive systems in the face of uncertainty. In Int Symp Search Based Software Engg, pp. 8195, Springer. DOI: 10.1007/978-3-642-39742-4_8CrossRefGoogle Scholar
Fredericks, E. M., DeVries, B., & Cheng, B. H. (2014). Towards run-time adaptation of test cases for self-adaptive systems in the face of uncertainty. In Proc 9th Int SEAMS symposium, pp. 1726. DOI: 10.1145/2593929.2593937Google Scholar
Filieri, A., Ghezzi, C., & Tamburrelli, G. (2011). Run-time efficient probabilistic model checking. In 33rd International Conference on Software Engineering (ICSE), pp. 341350. DOI: 10.1145/1985793.1985840Google Scholar
Flannery, A., Pena, M.A., Manns, J. (2018). Resilience in Transportation Planning, Engineering, Management, Policy, and Administration. Transportation Research Board, DOI:10.17226/25166Google Scholar
Gasser, P., Lustenberger, P., Cinelli, M., Kim, W., Spada, M., Burgherr, P., Hirschberg, S., Stojadinovic, B., Sun, T.Y., (2019). A review on resilience assessment of energy systems. Sustain. Resilient Infrastruct. 0, 127. DOI: 10.1080/23789689.2019.1610600Google Scholar
Holling, C.S., (1973). Resilience and Stability of Ecological Systems. Annu. Rev. Ecol. Syst. 4, 123. DOI: 10.1146/ Scholar
Hollnagel, E., Woods, D.D., Leveson, N., (2006). Resilience Engineering: Concepts and Precepts. Ashgate Publishing, Ltd.Google Scholar
Hollnagel, P.E., (2013). Resilience Engineering in Practice: A Guidebook. Ashgate Publishing, Ltd.Google Scholar
Jackson, S., Ferris, T.L.J., (2013). Resilience principles for engineered systems. Syst. Eng. 16, 152164. DOI: 10.1002/sys.21228CrossRefGoogle Scholar
Kabir, S., & Papadopoulos, Y. (2020). Computational Intelligence for Safety Assurance of Cooperative Systems of Systems. Computer, 53(12), 2434. DOI: 10.1109/MC.2020.3014604CrossRefGoogle Scholar
Kelly, T., & Weaver, R. (2004). The goal structuring notation–a safety argument notation. In Proceedings of the dependable systems and networks workshop on assurance cases, p. 16. DOI: Scholar
Knight, J., Rowanhill, J., & Xiang, J. (2014). A safety condition monitoring system. In International Conference on Computer Safety, Reliability, and Security, pp. 8394. DOI: 10.1007/978-3-319-24249-1_8Google Scholar
Kwiatkowska, M., Norman, G., & Parker, D. (2009). PRISM: probabilistic model checking for performance and reliability analysis. ACM SIGMETRICS Perf Eval Review, 36(4), 4045. DOI: 10.1145/1530873.1530882CrossRefGoogle Scholar
Mak, W.H.J, Clarkson, P.J. (2017). Towards the Design of Resilient Large-Scale Engineering Systems, Procedia CIRP, 60: 536541, doi: 10.1016/j.procir.2017.01.034.Google Scholar
McDermott, T.A. (2019). A Rigorous System Engineering Process for Resilient Cyber-Physical Systems Design. In 2019 Int Sym on Systems Engineering (ISSE), pp. 18, IEEE. DOI: 10.1109/ISSE46696.2019.8984569Google Scholar
Moura, J., Hutchinson, D., (2019) Cyber-physical systems resilience: state of the art, research issues and future trends, Computer science, Arxiv. Scholar
Nguyen, C. D., Miles, S., Perini, A., Tonella, P., Harman, M., & Luck, M. (2012). Evolutionary testing of autonomous software agents. Autonomous Agents and Multi-Agent Systems, 25(2), 260283. DOI: 10.1007/s10458-011-9175-4CrossRefGoogle Scholar
NIST (2016) National Institute for Standards and Technology (NIST) Framework for Cyber-Physical Systems Release 1.0: Cyber Physical Systems Public Working Group (Rep.). May 2016.Google Scholar
Rushby, J. (2008). Runtime certification. In International Workshop on Runtime Verification, pp. 2135, Springer, Berlin, Heidelberg. DOI: 10.1007/978-3-540-89247-2_2CrossRefGoogle Scholar
Schneider, D., & Trapp, M. (2013). Conditional safety certification of open adaptive systems. ACM Transactions on Autonomous and Adaptive Systems (TAAS), 8(2), 120. DOI: 10.1145/2491465.2491467CrossRefGoogle Scholar
Schneider, D., Trapp, M., Papadopoulos, Y., Armengaud, E., Zeller, M., & Höfig, K. (2015). WAP: digital dependability identities. In 2015 IEEE 26th ISSRE Symposium, pp. 324329. DOI: 10.1109/ISSRE.2015.7381825CrossRefGoogle Scholar
SaFAD (2019) Safety First for Automated Driving, white paper, retrieved from Scholar
United Nations, (2009). UNISDR Terminology and Disaster Risk Reduction.Google Scholar
Ushakov, I.A., (1994) Handbook of Reliability Engineering, Wiley & Sons. DOI:10.1002/9780470172414Google Scholar
Whittle, J., Sawyer, P., Bencomo, N., Cheng, B. H., & Bruel, J. M. (2009). Relax: Incorporating uncertainty into the specification of self-adaptive systems. In 17th IEEE International Requirements Engineering Conference, pp. 7988, IEEE. DOI: 10.1109/RE.2009.36CrossRefGoogle Scholar