Hostname: page-component-76fb5796d-2lccl Total loading time: 0 Render date: 2024-04-26T15:18:56.034Z Has data issue: false hasContentIssue false
  • English
  • Français

ONLINE CAPACITY PLANNING FOR REHABILITATION TREATMENTS: AN APPROXIMATE DYNAMIC PROGRAMMING APPROACH

Published online by Cambridge University Press:  11 December 2018

Ingeborg A. Bikker Open the ORCID record for Ingeborg A. Bikker [Opens in a new window] ,
Martijn R.K. Mes ,
Antoine Sauré  and
Richard J. Boucherie
Ingeborg A. Bikker
Affiliation:
Center for Healthcare Operations Improvement and Research (CHOIR), University of Twente Drienerlolaan 5, 7500 AE Enschede, The Netherlands and Department of Healthcare Logistics, Sint Maartenskliniek, Hengstdal 3, 6574 NA Ubbergen (Nijmegen), The Netherlands and Stochastic Operations Research, Department of Applied Mathematics, University of Twente, Drienerlolaan 5, 7500 AE Enschede, The Netherlands E-mail: i.a.bikker@utwente.nl
Martijn R.K. Mes
Affiliation:
Department Industrial Engineering and Business Information Systems (IEBIS) University of Twente Drienerlolaan 5, 7500 AE Enschede, The Netherlands
Antoine Sauré
Affiliation:
Telfer School of Management, University of Ottawa, 55 Laurier Avenue East, Ottawa, ON, Canada K1N 6N5
Richard J. Boucherie
Affiliation:
Center for Healthcare Operations Improvement and Research (CHOIR), University of Twente Drienerlolaan 5, 7500 AE Enschede, The Netherlands and Stochastic Operations Research, Department of Applied Mathematics, University of Twente, Drienerlolaan 5, 7500 AE Enschede, The Netherlands
Get access Rights & Permissions [Opens in a new window]

Abstract

We study an online capacity planning problem in which arriving patients require a series of appointments at several departments, within a certain access time target.

This research is motivated by a study of rehabilitation planning practices at the Sint Maartenskliniek hospital (the Netherlands). In practice, the prescribed treatments and activities are typically booked starting in the first available week, leaving no space for urgent patients who require a series of appointments at a short notice. This leads to the rescheduling of appointments or long access times for urgent patients, which has a negative effect on the quality of care and on patient satisfaction.

We propose an approach for allocating capacity to patients at the moment of their arrival, in such a way that the total number of requests booked within their corresponding access time targets is maximized. The model considers online decision making regarding multi-priority, multi-appointment, and multi-resource capacity allocation. We formulate this problem as a Markov decision process (MDP) that takes into account the current patient schedule, and future arrivals. We develop an approximate dynamic programming (ADP) algorithm to obtain approximate optimal capacity allocation policies. We provide insights into the characteristics of the optimal policies and evaluate the performance of the resulting policies using simulation.

Keywords

approximate dynamic programminghealthcare logisticsonline capacity planningoperations researchMarkov decision processrehabilitation treatment planningsimulation
Type
Research Article
Information
Probability in the Engineering and Informational Sciences , Volume 34 , Special Issue 3: Analytic Methods in Health Care and in Clinical Trials In Honor and Memory of Lycourgos (Lee) Papayanopoulos , July 2020 , pp. 381 - 405
Copyright
Copyright © Cambridge University Press 2018

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

1.Baharian, G. & Jacobson, S.H. (2013). Stochastic sequential assignment problem with threshold criteria. Probability in the Engineering and Informational Sciences 27(3): 277–296.CrossRefGoogle Scholar
2.Boucherie, R.J. & van Dijk, N.M. (eds.), (2017). International Series in Operations Research and Management Science: Markov decision processes in Practice. Cham, Switzerland: Springer.CrossRefGoogle Scholar
3.Braaksma, A., Kortbeek, N., Post, G.F. & Nollet, F. (2014). Integral multidisciplinary rehabilitation treatment planning. Operations Research for Health Care 3(3): 145–159.CrossRefGoogle Scholar
4.Cayirli, T. & Veral, E. (2003). Outpatient scheduling in health care: a review of literature. Production and Operations Management 12(4): 519–549.CrossRefGoogle Scholar
5.Dafoe, W., Arthur, H., Stokes, H., Morrin, L. & Beaton, L. (2006). Universal access: but when? Treating the right patient at the right time: access to cardiac rehabilitation. Canadian Journal of Cardiology 22: 905–911.CrossRefGoogle ScholarPubMed
6.De Farias, D. P. & Van Roy, B. (2003). The linear programming approach to approximate dynamic programming. Operations Research 51(6): 850–865.CrossRefGoogle Scholar
7.De Farias, D.P. & Van Roy, B. (2004). On constraint sampling in the linear programming approach to approximate dynamic programming. Mathematics of operations research 29(3): 462–478.CrossRefGoogle Scholar
8.Desaulniers, G., Desrosiers, J. & Solomon, M.M. (eds.), (2006). Column generation, vol. 5, New York, NY, USA: Springer Science & Business Media.Google Scholar
9.Erdelyi, A. & Topaloglu, H. (2010). Approximate dynamic programming for dynamic capacity allocation with multiple priority levels. IIE Transactions 43: 129–142.CrossRefGoogle Scholar
10.Fan, J. & Li, R. (2006). Statistical challenges with high dimensionality: feature selection in knowledge discovery. arXiv preprint math/0602133.Google Scholar
11.Gocgun, Y. & Ghate, A. (2012). Lagrangian relaxation and constraint generation for allocation and advanced scheduling. Computers & Operations Research 39: 2323–2336.CrossRefGoogle Scholar
12.Gocgun, Y. & Puterman, M.L. (2014). Dynamic scheduling with due dates and time windows: an application to chemotherapy patient appointment booking. Health Care Management Science 17: 60–76.CrossRefGoogle ScholarPubMed
13.Grötschel, M. & Holland, O. (1991). Solution of large-scale symmetric travelling salesman problems. Mathematical Programming 51: 141–202.CrossRefGoogle Scholar
14.Guestrin, C., Koller, D. & Parr, R. (2001). Max-norm projections for factored MDPs. In Seventeenth International Joint Conference on Artificial Intelligence, vol. 1, pp. 673–682.Google Scholar
15.Gupta, D. & Denton, B. (2008). Appointment scheduling in health care: Challenges and opportunities. IIE transactions 40(9): 800–819.CrossRefGoogle Scholar
16.Hulshof, P.J.H., Mes, M.R.K., Boucherie, R.J. & Hans, E.W. (2016). Patient admission planning using approximate dynamic programming. Flexible Services and Manufacturing 28: 30–61.CrossRefGoogle Scholar
17.Leeftink, A.G., Bikker, I.A., Vliegen, I.M.H. & Boucherie, R.J. (2018). Multi-disciplinary planning in health care: a review. Health Systems 1–24.Google Scholar
18.Master, N., Chan, C.W. & Bambos, N. (2018). Myopic policies for non-preemptive scheduling of jobs with decaying value. Probability in the Engineering and Informational Sciences 32(1): 1–36.CrossRefGoogle Scholar
19.Maxwell, M.S., Restrepo, M., Henderson, S.G. & Topaloglu, H. (2010). Approximate dynamic programming for ambulance redeployment. INFORMS Journal on Computing 22: 266–281.CrossRefGoogle Scholar
20.Mes, M. & Pérez Rivera, A. (2017). Approximate dynamic programming by practical examples. In Boucherie, R.J. & van Dijk, N.M. (eds.), Markov Decision Processes in Practice, chapter 3, Cham, Switzerland: Springer, pp. 61–101.Google Scholar
21.Patrick, J. & Puterman, M.L. (2007). Improving resource utilization for diagnostic services through flexible inpatient scheduling: a method for improving resource utilization. Journal of the Operational Research Society 58: 235–245.CrossRefGoogle Scholar
22.Patrick, J., Puterman, M.L. & Queyranne, M. (2008). Dynamic multipriority patient scheduling for a diagnostic resource. Operations Research 56: 1507–1525.CrossRefGoogle Scholar
23.Powell, W.B. (2011). Approximate Dynamic Programming: solving the curses of dimensionality, 2nd ed., Wiley Series in Probability and Statistics. New York, NY, USA: John Wiley & Sons. Inc.CrossRefGoogle Scholar
24.Pruhs, K., Sgall, J. & Torng, E. (2004). Online scheduling. In Leung, J.Y.-T. (ed.), Handbook of Scheduling: Algorithms, Models, and Performance Analysis. Boca Raton, FL: CRC Press, pp. 15-1–15-39.Google Scholar
25.Puterman, M.L. (2014). Markov decision processes: discrete stochastic dynamic programming. Hoboken, New Jersey, USA: John Wiley & Sons.Google Scholar
26. Revalidatie Nederland (Rehabilitation Sector Association The Netherlands), retr. Deccember 20, 2017. Utrecht, the Netherlands: Revalidatie Nederland. Available from http://www.revalidatie.nl/revalideren/home-r.Google Scholar
27.Roubos, D. & Bhulai, S. (2012). Approximate dynamic programming techniques for skill-based routing in call centers. Probability in the Engineering and Informational Sciences 26(4): 581–591.CrossRefGoogle Scholar
28.Sauré, A., Patrick, J., Tyldesley, S. & Puterman, M.L. (2012). Dynamic multi-appointment patient scheduling for radiation therapy. European Journal of Operational Research 223: 573–584.CrossRefGoogle Scholar
29.Scheer, J., Kroll, T., Neri, M.T. & Beatty, P. (2003). Access barriers for persons with disabilities. Journal of Disability Policy Studies 13: 221–230.CrossRefGoogle Scholar
30.Schimmelpfeng, K., Helber, S. & Kasper, S. (2012). Decision support for rehabilitation hospital scheduling. OR spectrum 34: 461–489.CrossRefGoogle Scholar
31.Schmid, V. (2012). Solving the dynamic ambulance relocation and dispatching problem using approximate dynamic programming. European Journal of Operational Research 219: 611–621.CrossRefGoogle ScholarPubMed
32.Schütz, H.J. & Kolisch, R. (2012). Approximate dynamic programming for capacity allocation in the service industry. European Journal of Operational Research 218: 239–250.CrossRefGoogle Scholar
33.Schütz, H.J. & Kolisch, R. (2013). Capacity allocation for demand of different customer-product-combinations with cancellations, no-shows, and overbooking when there is a sequential delivery of service. Annals of operations research 206: 401–423.CrossRefGoogle Scholar
34.Sgall, J. (1998). On-line scheduling. In Fiat, A. & Woeginger, G. (eds.), Online Algorithms, vol. 1442 of Lecture Notes in Computer Science, Berlin Heidelberg: Springer, pp. 196–231.Google Scholar
35.Simao, H. & Powell, W.B. (2009). Approximate dynamic programming for management of high-value spare parts. Journal of Manufacturing Technology Management 20: 147–160.CrossRefGoogle Scholar
36.Topaloglu, H. & Powell, W.B. (2006). Dynamic-programming approximations for stochastic time-staged integer multicommodity-flow problems. INFORMS Journal on Computing 18: 31–42.CrossRefGoogle Scholar
37.Tsitsiklis, J.N. & van Roy, B. (1996). Feature-based methods for large scale dynamic programming. Machine Learning 22(1–3): 59–94.CrossRefGoogle Scholar