Hostname: page-component-77c89778f8-5wvtr Total loading time: 0 Render date: 2024-07-18T21:59:59.299Z Has data issue: false hasContentIssue false


Published online by Cambridge University Press:  04 June 2021

Stéphane Loisel*
Univ Lyon, Université Claude Bernard Lyon 1 Institut de Science Financière et d’Assurances (ISFA) Laboratoire SAF EA2429, F-69366, Lyon, France
Pierrick Piette
Univ Lyon, Université Claude Bernard Lyon 1 Institut de Science Financière et d’Assurances (ISFA) Laboratoire SAF EA2429, F-69366, Lyon, France Seyna, 58 Rue de la Victoire, 75009 Paris, France E-Mail:
Cheng-Hsien Jason Tsai
Department of Risk Management and Insurance Risk and Insurance Research Center, College of Commerce National Chengchi University (NCCU)Taipei City, Taiwan E-Mail:


Modeling policyholders’ lapse behaviors is important to a life insurer, since lapses affect pricing, reserving, profitability, liquidity, risk management, and the solvency of the insurer. In this paper, we apply two machine learning methods to lapse modeling. Then, we evaluate the performance of these two methods along with two popular statistical methods by means of statistical accuracy and profitability measure. Moreover, we adopt an innovative point of view on the lapse prediction problem that comes from churn management. We transform the classification problem into a regression question and then perform optimization, which is new to lapse risk management. We apply the aforementioned four methods to a large real-world insurance dataset. The results show that Extreme Gradient Boosting (XGBoost) and support vector machine outperform logistic regression (LR) and classification and regression tree with respect to statistic accuracy, while LR performs as well as XGBoost in terms of retention gains. This highlights the importance of a proper validation metric when comparing different methods. The optimization after the transformation brings out significant and consistent increases in economic gains. Therefore, the insurer should conduct optimization on its economic objective to achieve optimal lapse management.

Research Article
© 2021 by Astin Bulletin. All rights reserved

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)


Albizzati, M.-O. and Geman, H. (1994) Interest rate risk management and valuation of the surrender option in life insurance policies. The Journal of Risk and Insurance, 61(4), 616637. CrossRefGoogle Scholar
Aleandri, M. (2017) Modeling dynamic policyholder behavior through machine learning techniques. Working paper. Google Scholar
Ascarza, E., Neslin, S.A., Netzer, O., Anderson, Z., Fader, P.S., Gupta, S., Hardie, B.G.S., Lemmens, A., Libai, B., Neal, D., Provost, F. and Schrift, R. (2018) In pursuit of enhanced customer retention management: Review, key issues, and future directions. Customer Needs and Solutions, 5(1), 6581. CrossRefGoogle Scholar
Bacinello, A.R. (2003) Pricing guaranteed life insurance participating policies with annual premiums and surrender option. North American Actuarial Journal, 7(3), 117. CrossRefGoogle Scholar
Barsotti, F., Milhaud, X. and Salhi, Y. (2016) Lapse risk in life insurance: Correlation and contagion effects among policyholders’ behaviors. Insurance: Mathematics and Economics, 71, 317331. Google Scholar
Bauer, D., Kiesel, R., Kling, A. and Ruß, J. (2006) Risk-neutral valuation of participating life insurance contracts. Insurance: Mathematics and Economics, 39(2), 171183. Google Scholar
Boser, B.E., Guyon, I.M. and Vapnik, V.N. (1992) A training algorithm for optimal margin classifiers. In Proceedings of the Fifth Annual Workshop on Computational Learning Theory, COLT 1992, pp. 144152. New York, NY, USA: ACM. CrossRefGoogle Scholar
Breiman, L. (2001) Random forests. Machine Learning, 45(1), 532. CrossRefGoogle Scholar
Breiman, L. (1996) Bagging predictors. Machine Learning, 24(2), 123140. CrossRefGoogle Scholar
Breiman, L., Friedman, J., Olshen, R. and Stone, C. (1984) Classification and Regression Trees. Wadsworth. CrossRefGoogle Scholar
Buchardt, K., Møller, T. and Schmidt, K.B. (2015) Cash flows and policyholder behaviour in the semi-Markov life insurance setup. Scandinavian Actuarial Journal, 8, 129. Google Scholar
Campbell, J., Chan, M., Li, K., Lombardi, L., Purushotham, M. and Rao, A. (2014) Modeling of policyholder behavior for life insurance and annuity products: A survey and literature review. Society of Actuaries. Google Scholar
Cerchiara, R.R., Gambini, A. and Edwards, M. (2009) Generalized linear models in life insurance: Decrements and risk factor analysis under solvency II. Giornale dell’Istituto Italiano degli Attuari, 72, 100122.Google Scholar
Chen, T. and Guestrin, C. (2016) XGBoost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2016, pp. 785794. New York, NY, USA: ACM. CrossRefGoogle Scholar
Chen, T., He, T., Benesty, M., Khotilovich, V. and Tang, Y. (2015) XGBoost: Extreme Gradient Boosting. R package Google Scholar
Consiglio, A. and Giovanni, D.D. (2010) Pricing the option to surrender in incomplete markets. Journal of Risk and Insurance, 77(4), 935957. CrossRefGoogle Scholar
Cortes, C. and Vapnik, V. (1995) Support-vector networks. Machine Learning, 20(3), 273297. CrossRefGoogle Scholar
Cox, S.H. and Lin, Y. (2006) Annuity lapse modeling: Tobit or not tobit? Society of Actuaries. Google Scholar
Dar, A. and Dodds, C. (1989) Interest rates, the emergency fund hypothesis and saving through endowment policies: Some empirical evidence for the U.K. The Journal of Risk and Insurance, 56(3), 415433. CrossRefGoogle Scholar
Eling, M. and Kiesenbauer, D. (2014) What policy features determine life insurance lapse? An analysis of the German market. Journal of Risk and Insurance, 81(2), 241269. CrossRefGoogle Scholar
Eling, M. and Kochanski, M. (2013) Research on lapse in life insurance: what has been done and what needs to be done? The Journal of Risk Finance, 14(4), 392413. CrossRefGoogle Scholar
European Insurance and Occupational Pensions Authority (EIOPA) (2011) EIOPA report on the Fifth Quantitative Impact Study (QIS5) for solvency II.Google Scholar
Friedman, J.H. (2002) Stochastic gradient boosting. Computational Statistics & Data Analysis, Nonlinear Methods and Data Mining, 38(4), 367378. CrossRefGoogle Scholar
Friedman, J.H. (2001) Greedy function approximation: A gradient boosting machine. The Annals of Statistics, 29(5), 11891232.CrossRefGoogle Scholar
Gatzert, N. and Schmeiser, H. (2008) Assessing the risk potential of premium payment options in participating life insurance contracts. Journal of Risk and Insurance, 75(3), 691712. CrossRefGoogle Scholar
Grosen, A. and Jørgensen, P.L. (2000) Fair valuation of life insurance liabilities: The impact of interest rate guarantees, surrender options, and bonus policies. Insurance: Mathematics and Economics, 26(1), 3757. Google Scholar
Gupta, S., Hanssens, D., Hardie, B., Kahn, W., Kumar, V., Lin, N., Ravishanker, N. and Sriram, S. (2006) Modeling customer lifetime value. Journal of Service Research, 9(2), 139155. CrossRefGoogle Scholar
Hoerl, A.E. and Kennard, R.W. (1970) Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12(1), 5567. CrossRefGoogle Scholar
Hsu, C.-W., Chang, C.-C. and Lin, C.-J. (2003) A practical guide to support vector classification. Working Paper. CrossRefGoogle Scholar
Hwang, Y. and Tsai, C. (2018) Differentiating surrender propensity from lapse propensity across life insurance products. Taiwan Risk Theory Seminar, Taipei, Taiwan.Google Scholar
Jamal, S. (2017) Lapse risk modeling with machine learning techniques: An application to structural lapse drivers. Bulletin Français d’Actuariat, 17(33): 2791.Google Scholar
Kagraoka, Y. (2005) Modeling insurance surrenders by the negative binomial model. Working paper. Google Scholar
Kim, C. (2005a) Surrender rate impacts on asset liability management. Asia-Pacific Journal of Risk and Insurance, 1(1). CrossRefGoogle Scholar
Kim, C. (2005b) Modeling surrender and lapse rates with economic variables. North American Actuarial Journal, 9(4), 5670. CrossRefGoogle Scholar
Kim, C. (2005c) Report to the policyholder behavior in the tail subgroups project. Society of Actuaries. Google Scholar
Kuo, W., Tsai, C. and Chen, W.K. (2003) An empirical study on the lapse rate: The cointegration approach. Journal of Risk and Insurance, 70(3), 489508. CrossRefGoogle Scholar
Lemmens, A. and Croux, C. (2006) Bagging and boosting classification trees to predict churn. Journal of Marketing Research, 43(2), 276286.CrossRefGoogle Scholar
Lemmens, A. and Gupta, S. (2020) Managing churn to maximize profits. Marketing Science, forthcoming.CrossRefGoogle Scholar
Loisel, S. and Milhaud, X. (2011) From deterministic to stochastic surrender risk models: Impact of correlation crises on economic capital. European Journal of Operational Research, 214(2), 348357. CrossRefGoogle Scholar
Meyer, D., Dimitriadou, E., Hornik, K., Leisch, F., Weingessel, A., Chang, C. and Lin, C. (2015) Misc Functions of Department of Statistics, Probability, Theory Group (Formerly: E1071). R package version 1.7-0.Google Scholar
Milhaud, X. (2013) Exogenous and endogenous risk factors management to predict surrender behaviours. ASTIN Bulletin: The Journal of the IAA, 43(3), 373398. CrossRefGoogle Scholar
Milhaud, X., Loisel, S. and Maume-Deschamps, V. (2011) Surrender triggers in Life Insurance: what main features affect the surrender behavior in a classical economic context? Bulletin Français d’Actuariat, 11(22), 548.Google Scholar
Nelder, J.A. and Wedderburn, R.W.M. (1972) Generalized linear models. Journal of the Royal Statistical Society: Series A (General), 135(3), 370384. CrossRefGoogle Scholar
Neslin, S.A., Gupta, S., Kamakura, W., Lu, J. and Mason, C.H. (2006) Defection detection: measuring and understanding the predictive accuracy of customer churn models. Journal of Marketing Research, 43(2), 204211. CrossRefGoogle Scholar
Nielsen, D. (2016) Tree boosting with XGBoost why does XGBoost win “Every” machine learning competition? (Master’s thesis), Norwegian University of Science and Technology, Trondheim. Scholar
Noble, W.S. (2006) What is a support vector machine? Nature Biotechnology, 24(12), 15651567. CrossRefGoogle ScholarPubMed
Pinquet, J., Guillén, M. and Ayuso, M. (2011) Commitment and lapse behavior in long-term insurance: A case study. Journal of Risk and Insurance, 78(4), 9831002. CrossRefGoogle Scholar
Powers, D.M. (2011) Evaluation: From precision, recall and F-measure to ROC, informedness, markedness and correlation. International Journal of Machine Learning Technology, 2(1), 3763.Google Scholar
Renshaw, A.E. and Haberman, S. (1986) Statistical analysis of life assurance lapses. Journal of the Institute of Actuaries, 113(3), 459497. CrossRefGoogle Scholar
Therneau, T., Aktinson, B. and Ripley, B. (2018) Recursive Partitioning and Regression Trees. R package version 4.1-13.Google Scholar
Tibshirani, R. (1996) Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society. Series B (Methodological), 58(1), 267288.CrossRefGoogle Scholar
Tsai, C., Kuo, W. and Chiang, D.M.-H. (2009) The distributions of policy reserves considering the policy-year structures of surrender rates and expense ratios. Journal of Risk and Insurance, 76(4), 909931. CrossRefGoogle Scholar
Vafeiadis, T., Diamantaras, K.I., Sarigiannidis, G. and Chatzisavvas, K.C. (2015) A comparison of machine learning techniques for customer churn prediction. Simulation Modelling Practice and Theory, 55, 19. CrossRefGoogle Scholar
Wainer, J. (2016) Comparison of 14 different families of classification algorithms on 115 binary datasets. Working paper. Google Scholar
Xia, G. and Jin, W. (2008) Model of customer churn prediction on support vector machine. Systems Engineering - Theory & Practice, 28(1), 7177. S1874-86510960003-XS1874-86510960003-XCrossRefGoogle Scholar
Zhao, Y., Li, B., Li, Xiu, Liu, W. and Ren, S. (2005) Customer churn prediction using improved one-class support vector machine. In: Advanced Data Mining and Applications (eds. Li, X., Wang, S. and Dong, Z.Y.), Lecture Notes in Computer Science, pp. 300306. Berlin, Heidelberg: Springer.CrossRefGoogle Scholar