Skip to main content Accessibility help

Development of a Parallel Explicit Finite-Volume Euler Equation Solver using the Immersed Boundary Method with Hybrid MPI-CUDA Paradigm

  • F. A. Kuo (a1) (a2), C. H. Chiang (a3), M. C. Lo (a4) and J. S. Wu (a1)


This study proposed the application of a novel immersed boundary method (IBM) for the treatment of irregular geometries using Cartesian computational grids for high speed compressible gas flows modelled using the unsteady Euler equations. Furthermore, the method is accelerated through the use of multiple Graphics Processing Units – specifically using Nvidia’s CUDA together with MPI - due to the computationally intensive nature associated with the numerical solution to multi-dimensional continuity equations. Due to the high degree of locality required for efficient multiple GPU computation, the Split Harten-Lax-van-Leer (SHLL) scheme is employed for vector splitting of fluxes across cell interfaces. NVIDIA visual profiler shows that our proposed method having a computational speed of 98.6 GFLOPS and 61% efficiency based on the Roofline analysis that provides the theoretical computing speed of reaching 160 GLOPS with an average 2.225 operations/byte. To demonstrate the validity of the method, results from several benchmark problems covering both subsonic and supersonic flow regimes are presented. Performance testing using 96 GPU devices demonstrates a speed up of 89 times that of a single GPU (i.e. 92% efficiency) for a benchmark problem employing 48 million cells. Discussions regarding communication overhead and parallel efficiency for varying problem sizes are also presented.


Corresponding author

*Corresponding author (


Hide All
1. Kamil, A. and Yelick, K., “Hierarchical Computation in the SPMD Programming Model,Languages and Compilers for Parallel Computing SE - 1, vol. 8664, pp. 319, (2014).
2. Fluent, A., “ANSYS Fluent 12.0 user’s guide,Ansys Inc, vol. 15317, pp. 12498, (2009).
3. Rodio, J. J., Xiao, X., Hassan, H. A., and Rumsey, C. L., “NASA Trapezoidal Wing Simulation using Stress-ω and One- and Two-Equation Turbulence Models,” Proceedings of AIAA SciTech 52nd Aerospace Sciences Meeting, pp. 119, (2014).
4. Jasak, H., Jemcov, A., and Tukovic, Z., “OpenFOAM : A C++ Library for Complex Physics Simulations,” Proceedings of International Workshop on Coupled Methods in Numerical Dynamics, pp. 120, (2007).
5. Phillips, E., Zhang, Y., Davis, R., and Owens, J., “Rapid aerodynamic performance prediction on a cluster of graphics processing units,” Proceedings of the 47th AIAA Aerospace Sciences Meeting, pp. 111, (2009).
6. Scheidegger, C. E., Comba, J. L. D., and da Cunha, R. D., “Practical CFD Simulations on Programmable Graphics Hardware using SMAC+,Computer Graphics Forum, vol. 24, no. 4, pp. 715728, (Dec. 2005).
7. Ma, Z. H., Wang, H., and Pu, S. H., “GPU computing of compressible flow problems by a meshless method with space-filling curves,Journal of Computational Physics, vol. 263, pp. 113135, (Apr. 2014).
8. Shahbazi, K., Hesthaven, J. S., and Zhu, X., “Multidimensional hybrid Fourier continuation–WENO solvers for conservation laws,Journal of Computational Physics, vol. 253, pp. 209225, (2013).
9. Tuttafesta, M., Colonna, G., and Pascazio, G., “Computing unsteady compressible flows using Roe’s flux-difference splitting scheme on GPUs,Computer Physics Communications, vol. 184, no. 6, pp. 14971510, (2013).
10. Niemeyer, K. E. and Sung, C. J., “Recent progress and challenges in exploiting graphics processors in computational fluid dynamics,The Journal of Supercomputing, vol. 67, no. 2, pp. 528564, (Feb. 2014).
11. Kuo, F. A., Smith, M. R., Hsieh, C. W., Chou, C. Y., and Wu, J. S., “GPU acceleration for general conservation equations and its application to several engineering problems,Computers & Fluids, vol. 45, pp. 147154, (2011).
12. Su, C. C., Smith, M. R., Kuo, F. A., Wu, J. S., Hsieh, C. W., and Tseng, K. C., “Large-scale simulations on multiple Graphics Processing Units (GPUs) for the direct simulation Monte Carlo method,Journal of Computational Physics, vol. 231, pp. 79327958, (2012).
13. Lo, M. C., Su, C. C., Wu, J. S., and Kuo, F. A., “Development of parallel direct simulation Monte Carlo method using a cut-cell Cartesian grid on a single graphics processor,Computers & Fluids, vol. 101. pp. 114125, (2014).
14. Schneider, E. E. and Robertson, B. E., “Cholla: A New Massively Parallel Hydrodynamics Code For Astrophysical Simulation,The Astrophysical Journal Supplement Series, vol. 217, no. 2, p. 24, (Apr. 2015).
15. Miao, S., Zhang, X., Parchment, O. G., and Chen, X., “A fast GPU based bidiagonal solver for computational aeroacoustics,Computer Methods in Applied Mechanics and Engineering, vol. 286, pp. 2239, (2015).
16. Zimmerman, B. J. and Wang, Z. J., “The efficient implementation of correction procedure via reconstruction with graphics processing unit computing,Computers & Fluids, vol. 101, pp. 263272, (Sep. 2014).
17. Fu, L., Gao, Z., Xu, K., and Xu, F., “A multi-block viscous flow solver based on GPU parallel methodology,Computers & Fluids, vol. 95, pp. 1939, (May 2014).
18. Shinn, A. F., Goodwin, M. A., and Vanka, S. P., “Immersed boundary computations of shear- and buoyancy-driven flows in complex enclosures,International Journal of Heat and Mass Transfer, vol. 52, no. 17–18, pp. 40824089, (2009).
19. Harten, A., Lax, P., and Leer, B., “On Upstream Differencing and Godunov-Type Schemes for Hyperbolic Conservation Laws,SIAM Review, vol. 25, no. 1, pp. 3561, (Jan. 1983).
20. Peskin, C. S., “Flow patterns around heart valves: A numerical method,Journal of Computational Physics, vol. 10, pp. 252271, (1972).
21. Mittal, R. and Iaccarino, G., “Immersed Boundary Methods,Annual Review of Fluid Mechanics, vol. 37. pp. 239261, (2005).
22. Clarke, D. K., Hassan, H. A., and Salas, M. D., “Euler calculations for multielement airfoils using Cartesian grids,AIAA Journal, vol. 24, no. 3, pp. 353358, (Mar. 1986).
23. Tseng, Y. H. and Ferziger, J. H., “A ghost-cell immersed boundary method for flow in complex geometry,Journal of Computational Physics, vol. 192, pp. 593623, (2003).
24. Kontzialis, K. and Ekaterinaris, J. A., “High order discontinuous Galerkin discretizations with a new limiting approach and positivity preservation for strong moving shocks,Computers & Fluids , vol. 71, pp. 98112, (2013).
25. Liu, J., Zhao, N., and Hu, O., “The ghost cell method and its applications for inviscid compressible flow on adaptive tree cartesian grids,Advances in Applied Mathematics and Mechanics, vol. 1, pp. 664682, (2009).
26. Ghias, R., Mittal, R., and Dong, H., “A sharp interface immersed boundary method for compressible viscous flows,Journal of Computational Physics, vol. 225, pp. 528553, (2007).
27. Marshall, D. D. and Ruffin, S. M., “An Embedded Boundary Cartesian Grid Scheme for Viscous Flows using a New Viscous Wall Boundary Condition Treatment,” Proceedings of 42nd AIAA Aerospace Sciences Meeting and Exhibit, pp. 18, (2004).
28. Colella, P., Graves, D. T., Keen, B. J., and Modiano, D., “A Cartesian grid embedded boundary method for hyperbolic conservation laws,Journal of Computational Physics, vol. 211, pp. 347366, (2006).
29. Toro, E. F., Riemann solvers and numerical methods for fluid dynamics-A Practical Introduction, (2009).
30. Davis, S. F., “Simplified Second-Order Godunov-Type Methods,SIAM Journal on Scientific and Statistical Computing, vol. 9, no. 3, pp. 445473, (May 1988).
31. Burden, R. L. and Faires, J. D., Numerical analysis. Brooks/Cole, (2001).
32. Williams, S. W., “The Roofline Model,” Performance Tuning of Scientific Applications, pp. 195215, (2011).
33. Wermelinger, F., Hejazialhosseini, B., Hadjidoukas, P., Rossinelli, D., and Koumoutsakos, P., “An Efficient Compressible Multicomponent Flow Solver for Heterogeneous CPU/GPU Architectures,” Proceedings of the Platform for Advanced Scientific Computing Conference on ZZZ - PASC ’16, (2016).
34. Williams, S., Waterman, A., and Patterson, D., “Roofline: an insightful visual performance model for multicore architectures,Communications of the ACM, vol. 52, pp. 6576, (2009).
35. Demirdžić, I., Lilek, Ž., and Perić, M., “A collocated finite volume method for predicting flows at all speeds,International Journal for Numerical Methods in fluids, vol. 16, no. January, pp. 10291050, (1993).
36. Furmánek, P., “Numerical Solution of Steady and Unsteady Compressible Flow,” Czech Technical University in Prague, (2008).
37. Chang, S. C., Wang, X. Y., and Chow, C. Y., “The Space-Time Conservation Element and Solution Element Method: A New High-Resolution and Genuinely Multidimensional Paradigm for Solving Conservation Laws,Journal of Computational Physics, vol. 136, pp. 89136, (1998).
38. Sun, M. and Takayama, K., “Vorticity production in shock diffraction,Journal of Fluid Mechanics, vol. 478, pp. 237256, (2003).


Development of a Parallel Explicit Finite-Volume Euler Equation Solver using the Immersed Boundary Method with Hybrid MPI-CUDA Paradigm

  • F. A. Kuo (a1) (a2), C. H. Chiang (a3), M. C. Lo (a4) and J. S. Wu (a1)


Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed