Hostname: page-component-848d4c4894-2xdlg Total loading time: 0 Render date: 2024-06-28T12:19:32.810Z Has data issue: false hasContentIssue false
  • English
  • Français

DETECTING AND CHARACTERIZING PATTERNS OF FAILURE IN COMPLEX ENGINEERED SYSTEMS: AN ONTOLOGY DEVELOPMENT AND CLUSTERING APPROACH

Published online by Cambridge University Press:  19 June 2023

Hannah Scharline Walsh ,
Andy Dong ,
Irem Tumer  and
Guillaume Brat
Hannah Scharline Walsh*
Affiliation:
NASA Ames Research Center
Andy Dong
Affiliation:
Oregon State University
Irem Tumer
Affiliation:
Oregon State University
Guillaume Brat
Affiliation:
NASA Ames Research Center
*
Walsh, Hannah Scharline, NASA Ames Research Center, United States of America, hannah.s.walsh@nasa.gov

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

While the causes of failures in complex engineered systems are often clear in hindsight, it can be challenging to predict failures proactively during the design of novel engineered products or systems. Identifying patterns can be useful for capturing common characteristics that may lead to failure. In this paper, we present a methodology for identifying patterns of failure from NASA's publicly available Lessons Learned Information System (LLIS). We apply an ontology development and clustering approach to identify representative patterns leading to failures in historical lessons learned. A joint inductive-deductive approach reveals the key themes in lessons that lead to failure, which are formalized and recorded as an ontology of complex systems failure causes. Documents from the LLIS are manually tagged with relevant characteristics from the ontology. From the tagged set, clustering is used to capture co-occurring sets of characteristics that lead to failure. The primary contribution of this work is a method for extracting a set of generic failure patterns in complex engineered systems and characteristics for these patterns that can be identified at design time, knowledge of which can be used to plan mitigation strategies.

Keywords

ComplexityRisk managementOntologies
Type
Article
Information
Proceedings of the Design Society , Volume 3: ICED23 , July 2023 , pp. 1425 - 1434
Creative Commons
Creative Common License - CCCreative Common License - BYCreative Common License - NCCreative Common License - ND
This is an Open Access article, distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives licence (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is unaltered and is properly cited. The written permission of Cambridge University Press must be obtained for commercial re-use or in order to create a derivative work.
Copyright
The Author(s), 2023. Published by Cambridge University Press

References

Al-Ahmad, W., Al-Fagih, K., Khanfar, K., Alsamara, K., Abuleil, S., and Abu-Salem, H., (2009), “A taxonomy of an IT project failure: root causes”, International Management Review, Vol. 5 No. 1, pp. 93–104, 106.Google Scholar
Andrade, S. R. and Walsh, H. S. (2022), “Discovering a failure taxonomy for early design of complex engineered systems using natural language processing”, Journal of Computing and Information Science in Engineering, Vol. 23 No. 3, p. 031001. https://dx.doi.org/10.1115/1.4054688.CrossRefGoogle Scholar
Andrade, S. R. and Walsh, H. S. (2022), “What went wrong: a survey of wildfire UAS mishaps using named entity recognition”, 2022 IEEE/AIAA 41st Digital Avionics Systems Conference (DASC), Portsmouth, VA, USA, IEEE. https://dx.doi.org/10.1109/DASC52595.2021.9594501.CrossRefGoogle Scholar
Bahill, T. (2012), “Diogenes, a process for identifying unintended consequences”, Systems Engineering, Vol. 15 No. 3, pp. 287–306. https://dx.doi.org/10.1002/sys.20208.CrossRefGoogle Scholar
Favi, C., Germani, M., Mandolini, M. and Marconi, M. (2016), “Disassembly knowledge classification and potential application: a preliminary analysis on a washing machine”, ASME 2016 International Design Engineering Technical Conferences and Computers & Information in Engineering Conference (IDETC/CIE 2016), Charlotte, North Carolina, USA, ASME, pp. V004T05A011. https://dx.doi.org/10.1115/detc2016-59514.Google Scholar
Gurnani, A. and Lewis, K. (2008), “Collaborative, decentralized engineering design at the edge of rationality”, Journal of Mechanical Design, Vol. 130 No. 12, p. 121101. https://dx.doi.org/10.1115/1.2988479.CrossRefGoogle Scholar
Hermes, S., Kaufmann-Ludwig, J., Schreieck, M., Weking, J., and Böhm, M., (2020), “A taxonomy of platform envelopment: revealing patterns and particularities”, AMCIS 2020 Proceedings.Google Scholar
Hsieh, H.-F. and Shannon, S. E. (2005), “Three approaches to qualitative content analysis”, Qualitative Health Research, Vol. 15 No. 9, pp. 1277–1288. https://dx.doi.org/10.1177/1049732305276687.CrossRefGoogle ScholarPubMed
Li, L., Qin, F., Gao, S. and Liu, Y. (2014), “An approach for design rationale retrieval using ontology-aided indexing”, Journal of Engineering Design, Vol. 25 No. 7-9, pp. 259–279. https://dx.doi.org/10.1115/detc2013-12522.CrossRefGoogle Scholar
Marais, K., Saleh, J. and Leveson, N. (2006), “Archetypes for organizational safety”, Safety Science, Vol. 44 No. 7, pp. 565–582. https://dx.doi.org/10.1016/j.ssci.2005.12.004.CrossRefGoogle Scholar
Mazzurco, A., Leydens, J. A. and Jesiek, B. K. (2018), “Passive, consultative, and coconstructive methods: a framework to facilitate community participation in design for development”, Journal of Mechanical Design, Vol. 140 No. 12, p. 121401. https://dx.doi.org/10.1115/1.4041171.Google Scholar
Ming, Z., Wang, G., Yan, Y.; Panchal, J. H., Goh, C., Allen, J. K., and Mistree, F. (2018), “Ontology-based representation of design decision hierarchies”, Journal of Computing and Information Systems in Engineering, Vol. 18 No. 1, p. 011001. https://dx.doi.org/10.1115/1.4037934.Google Scholar
NASA, NASA Public Lessons Learned Information System. [online] Available at: https://llis.nasa.gov/ [Accessed 2022].Google Scholar
Nickerson, R. C., Varshney, U. and Muntermann, J. (2013), “A method for taxonomy development and its application in information systems”, European Journal of Information Systems, Vol. 22, pp. 336–359. https://dx.doi.org/10.1057/ejis.2012.26.CrossRefGoogle Scholar
O'Halloran, B. M., Stone, R. B. and Tumer, I. Y. (2012), “A failure modes and mechanisms naming taxonomy”, IEEE 2012 Proceedings Annual Reliability and Maintainability Symposium, IEEE, pp. 1–6. https://dx.doi.org/10.1109/rams.2012.6175455.Google Scholar
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., Duchesnay, E. (2011), “Scikit-learn: machine learning in Python”, Journal of Machine Learning Research, Vol. 12, pp. 2825–2830.Google Scholar
Sai, A. R., Buckley, J., Fitzgerald, B. and Andrew, L. G. (2021), “Taxonomy of centralization in public blockchain systems: A systematic literature review”, Information Processing & Management, Vol. 58 No. 4, p. 102584. https://dx.doi.org/10.1016/j.ipm.2021.102584.CrossRefGoogle Scholar
Sarkar, A. a. S. D. (2016), “Foundation ontology for distributed manufacturing process planning”, ASME 2016 International Design Engineering Technical Conferences and Computers & Information in Engineering Conference (IDETC/CIE 2016), Charlotte, North Carolina, USA, pp. V01BT02A031. https://dx.doi.org/10.1115/detc2016-60159.Google Scholar
Scharfe, P. and Wiener, M. (2020), “A taxonomy of smart machines in the mechanical engineering industry: toward structuring the design solution space”, ICIS 2020 Proceedings, p. 1139.Google Scholar
Shi, F., Chen, L., Han, J. and Childs, P. (2017), “A data-driven text mining and semantic network analysis for design information retrieval”, Journal of Mechanical Design, Vol. 139 No. 11, p. 111402. https://dx.doi.org/10.1115/1.4037649.Google Scholar
Sowa, J. F. and Zachman, J. A. (1992), “Extending and formalizing the framework for information systems architecture”, IBM Systems Journal, Vol. 31 No. 3, pp. 590–616. https://dx.doi.org/10.1147/sj.313.0590.CrossRefGoogle Scholar
Stone, R. B., Tumer, I. Y. and Van Wie, M. (2005), “The function-failure design method”, Journal of Mechanical Design, Vol. 127, pp. 397–407. https://dx.doi.org/10.1115/1.1862678.CrossRefGoogle Scholar
Sutcliffe, A. and Rugg, G. (1998), “A taxonomy of error types for failure analysis and risk assessment”, Int. J. Hum. Comput. Interaction, Vol. 10 No. 4, pp. 381–405. https://dx.doi.org/10.1207/s15327590ijhc1004_5.CrossRefGoogle Scholar
von Luxburg, U. (2007), “A tutorial on spectral clustering”, Statistics and Computing, Vol. 17 No. 4, pp. 395–416. https://dx.doi.org/10.1007/s11222-007-9033-z.CrossRefGoogle Scholar
Wang, K., Stevens, R., Alachram, H., Li, Y., Soldatova, L., King, R., Ananiadou, S., Schoene, A. M., Li, Mao., and Christopoulou, F. (2021), “NERO: a biomedical named-entity (recognition) ontology with a large, annotated corpus reveals meaningful associations through text embedding”, NPJ systems biology and applications, Vol. 7 No. 1, pp. 1–8. https://dx.doi.org/10.1101/2020.11.05.368969.CrossRefGoogle Scholar
Weking, J., Mandalenakis, M., Hein, A., Hermes, S., Böhm, M., and Krcmar, H. (2020), “The impact of blockchain technology on business models –- a taxonomy and archetypal patterns”, Electronic Markets, Vol. 30 No. 2, pp. 285–305. https://dx.doi.org/10.1007/s12525-019-00386-3.Google Scholar
Wolstenholme, E. F. (2003), “Towards the definition and use of a core set of archetypal structures in system dynamics”, System Dynamics Review, Vol. 9 No. 1, pp. 7–26. https://dx.doi.org/10.1002/sdr.259.CrossRefGoogle Scholar
Ye, D., Xing, Z., Foo, C., Ang, Z., Li, J., and Kapre, N. (2016), “Software-specific named entity recognition in software engineering social content”, 2016 IEEE 23rd International Conference on Software Analysis, Evolution and Reengineering (SANER), IEEE, pp. 90–101. https://dx.doi.org/10.1109/saner.2016.10.CrossRefGoogle Scholar
Zhang, C., Zhou, G., Bai, Q., Lu, Q., and Chang, F. (2018), “HEKM: a high-end equipment knowledge management system for supporting knowledge-driven decision-making in new product development”, ASME 2018 International Design Engineering Technical Conferences and Computers & Information in Engineering Conference (IDETC/CIE 2018), ASME, Quebec City, Quebec, Canada, pp. V01BT02A014. https://dx.doi.org/10.1115/detc2018-85151.Google Scholar