Skip to main content Accessibility help

How to compare performance of robust design optimization algorithms, including a novel method

  • Johan A. Persson (a1) and Johan Ölvander (a1)


This paper proposes a method to compare the performances of different methods for robust design optimization of computationally demanding models. Its intended usage is to help the engineer to choose the optimization approach when faced with a robust optimization problem. This paper demonstrates the usage of the method to find the most appropriate robust design optimization method to solve an engineering problem. Five robust design optimization methods, including a novel method, are compared in the demonstration of the comparison method. Four of the five compared methods involve surrogate models to reduce the computational cost of performing robust design optimization. The five methods are used to optimize several mathematical functions that should be similar to the engineering problem. The methods are then used to optimize the engineering problem to confirm that the most suitable optimization method was identified. The performance metrics used are the mean value and standard deviation of the robust optimum as well as an index that combines the required number of simulations of the original model with the accuracy of the obtained solution. These measures represent the accuracy, robustness, and efficiency of the compared methods. The results of the comparison show that sequential robust optimization is the method with the best balance between accuracy and number of function evaluations. This is confirmed by the optimizations of the engineering problem. The comparison also shows that the novel method is better than its predecessor is.


Corresponding author

Reprint requests to: Johan A. Persson, Department of Management and Engineering, Linköping University, Linköping SE-581 83, Sweden. E-mail:


Hide All
Aspenberg, D., Jergeus, J., & Nilsson, L. (2013). Robust optimization of front members in a full frontal car impact. Engineering Optimization 45(3), 245264.
Beyer, H.G., & Sendhoff, B. (2007). Robust optimization—a comprehensive survey. Computer Methods in Applied Mechanics and Engineering 196(33), 31903218.
Box, M.J. (1965). A new method of constrained optimization and a comparison with other methods. Computer Journal 8(1), 4252.
Branke, J. (2001). Evolutionary Optimization in Dynamic Environments. Norwell, MA: Kluwer Academic.
Coelho, R.F. (2014). Metamodels for mixed variables based on moving least squares. Optimization and Engineering 15(2), 311329.
Eberhart, R.C., & Kennedy, J. (1995). A new optimizer using particle swarm theory. Proc. 6th Int. Symp. Micro Machine and Human Science, Vol. 1, pp. 3943, Nagoya, Japan, October 4–6.
Forrester, A., Sobester, A., & Keane, A. (2008). Engineering Design Via Surrogate Modelling: A Practical Guide. Hoboken, NJ: Wiley.
Gobbi, M., Guarneri, P., Scala, L., & Scotti, L. (2014). A local approximation based multi-objective optimization algorithm with applications. Optimization and Engineering 15(3), 619641.
Goldberg, D.E. (2006). Genetic Algorithms. Delhi: Pearson Education India.
Holland, J.H. (1975). Adaptation in Natural and Artificial Systems: An Introductory Analysis With Applications to Biology, Control, and Artificial Intelligence. Ann Arbor, MI: University of Michigan Press.
Jin, R., Du, X., & Chen, W. (2003). The use of metamodeling techniques for optimization under uncertainty. Structural and Multidisciplinary Optimization 25(2), 99116.
Jones, D.R. (2001). A taxonomy of global optimization methods based on response surfaces. Journal of Global Optimization 21(4), 345383.
Krus, P., & Ölvander, J. (2013). Performance index and meta-optimization of a direct search optimization method. Engineering Optimization 45(10), 11671185.
McKay, M.D., Beckman, R.J., & Conover, W.J. (1979). Comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 21(2), 239245.
Mercer, R.E., & Sampson, J.R. (1978). Adaptive search using a reproductive metaplan. Kybernetes 7(3), 215228. doi:10.1108/eb005486
Myers, R.H., Montgomery, D.C., & Anderson-Cook, C.M. (2009). Response Surface Methodology: Process and Product Optimization Using Designed Experiments. Hoboken, NJ: Wiley.
Neculai, A. (2008). An unconstrained optimization test functions collection. Advanced Modeling and Optimization 10(1), 147161.
Nelder, J.A., & Mead, R. (1965). A simplex method for function minimization. Computer Journal 7(4), 308313.
Ölvander, J., & Krus, P. (2006). Optimizing the optimization—a method for comparison of optimization algorithms. Proc. AIAA Multidisciplinary Design Optimization Specialists Conf., Paper No. AIAA 2006-1915, Newport, RI, May 1–4. doi:10.2514/6.2006-1915
Paenke, I., Branke, J., & Jin, Y. (2006). Efficient search for robust solutions by means of evolutionary algorithms and fitness approximation. IEEE Transactions on Evolutionary Computation 10(4), 405420.
Persson, J., & Ölvander, J. (2011). Comparison of sampling methods for a dynamic pressure regulator. Proc. 49th AIAA Aerospace Sciences Meeting, Paper No. AIAA 2011-1205, Orlando, FL, January 4–7. doi:10.2514/6.2011-1205
Persson, J.A., & Ölvander, J. (2013). Comparison of different uses of metamodels for robust design optimization. Proc. 51th AIAA Aerospace Sciences Meeting, Paper No. AIAA 2013-1039, Grapevine, TX, January 7–10. doi:10.2514/6.2013-1039
Persson, J.A., & Ölvander, J. (2015). Optimization of the complex-RFM optimization algorithm. Optimization and Engineering 16(1), 2748.
Rehman, S., Langelaar, M., & van Keulen, F. (2014). Efficient kriging-based robust optimization of unconstrained problems. Journal of Computational Science. Advance online publication. doi:10.1016/j.jocs.2014.04.005
Reisenthel, P.H., & Lesieutre, D. J. (2011). A numerical experiment on allocating resources between design of experiment samples and surrogate-based optimization infills. Proc. 52ndAIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics and Materials Conf., Paper No. AIAA 2011-2150, Denver, CO, April, 4–7.
Schutte, J.F., & Haftka, R.T. (2005). Improved global convergence probability using independent swarms. Proc. 46th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics and Materials Conf., Structures, Structural Dynamics, and Materials and Co-located Conf., Austin, TX. doi:10.2514/6.2005-1896
Tarkian, M., Persson, J., Ölvander, J., & Feng, X. (2012). Multidisciplinary design optimization of modular industrial robots by utilizing high level CAD templates. Journal of Mechanical Design 134(12), 124502.
Tenne, Y. (2015). An adaptive-topology ensemble algorithm for engineering optimization problems. Optimization and Engineering 16(2), 303334.
Toal, D.J.J., & Keane, A.J. (2012). Non-stationary kriging for design optimization. Engineering Optimization 44(6), 741765.
Wang, G.G., & Shan, S. (2007). Review of metamodeling techniques in support of engineering design optimization. Journal of Mechanical Design 129(4), 370380.
Wiebenga, J.H., Van Den Boogaard, A.H., & Klaseboer, G. (2012). Sequential robust optimization of a V-bending process using numerical simulations. Structural and Multidisciplinary Optimization 46(1), 137153.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

  • ISSN: 0890-0604
  • EISSN: 1469-1760
  • URL: /core/journals/ai-edam
Please enter your name
Please enter a valid email address
Who would you like to send this to? *



Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed