Skip to main content Accessibility help

An introduction to continuous optimization for imaging

  • Antonin Chambolle (a1) and Thomas Pock (a2)

A large number of imaging problems reduce to the optimization of a cost function, with typical structural properties. The aim of this paper is to describe the state of the art in continuous optimization methods for such problems, and present the most successful approaches and their interconnections. We place particular emphasis on optimal first-order schemes that can deal with typical non-smooth and large-scale objective functions used in imaging problems. We illustrate and compare the different algorithms using classical non-smooth problems in imaging, such as denoising and deblurring. Moreover, we present applications of the algorithms to more advanced problems, such as magnetic resonance imaging, multilabel image segmentation, optical flow estimation, stereo matching, and classification.

  • View HTML
    • Send article to Kindle

      To send this article to your Kindle, first ensure is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

      Note you can select to send to either the or variations. ‘’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

      Find out more about the Kindle Personal Document Service.

      An introduction to continuous optimization for imaging
      Available formats
      Send article to Dropbox

      To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

      An introduction to continuous optimization for imaging
      Available formats
      Send article to Google Drive

      To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

      An introduction to continuous optimization for imaging
      Available formats
Hide All
Aharon, M., Elad, M. and Bruckstein, A. (2006), ‘K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation’, IEEE Trans. Signal Process. 54, 43114322.
Ahuja, R. K., Magnanti, T. L. and Orlin, J. B. (1993), Network Flows: Theory, Algorithms, and Applications, Prentice Hall.
Aĭzerman, M. A., Braverman, È. M. and Rozonoèr, L. I. (1964), ‘A probabilistic problem on automata learning by pattern recognition and the method of potential functions’, Avtomat. i Telemeh. 25, 13071323.
Alberti, G., Bouchitté, G. and Dal Maso, G. (2003), ‘The calibration method for the Mumford–Shah functional and free-discontinuity problems’, Calc. Var. Partial Differential Equations 16, 299333.
Allen-Zhu, Z. and Orecchia, L. (2014), Linear coupling: An ultimate unification of gradient and mirror descent. arXiv:1407.1537
Almeida, M. and Figueiredo, M. A. T. (2013), ‘Deconvolving images with unknown boundaries using the alternating direction method of multipliers’, IEEE Trans. Image Process. 22, 30743086.
Alvarez, F. (2003), ‘Weak convergence of a relaxed and inertial hybrid projection-proximal point algorithm for maximal monotone operators in Hilbert space’, SIAM J. Optim. 14, 773782.
Alvarez, F. and Attouch, H. (2001), ‘An inertial proximal method for maximal monotone operators via discretization of a nonlinear oscillator with damping’, Set-Valued Analysis 9, 311.
Ambrosio, L. and Masnou, S. (2003), ‘A direct variational approach to a problem arising in image reconstruction’, Interfaces Free Bound. 5, 6381.
Ambrosio, L. and Tortorelli, V. M. (1992), ‘On the approximation of free discontinuity problems’, Boll. Un. Mat. Ital. B 6(7), 105123.
Ambrosio, L., Fusco, N. and Pallara, D. (2000), Functions of Bounded Variation and Free Discontinuity Problems, The Clarendon Press, Oxford University Press.
Arias, P., Facciolo, G., Caselles, V. and Sapiro, G. (2011), ‘A variational framework for exemplar-based image inpainting’, Internat. J. Computer Vision 93, 319347.
Armijo, L. (1966), ‘Minimization of functions having Lipschitz continuous first partial derivatives’, Pacific J. Math. 16, 13.
, Arrow, K. J., Hurwicz, L. and Uzawa, H. (1958), Studies in Linear and Non-linear Programming, Vol. II of Stanford Mathematical Studies in the Social Sciences, Stanford University Press.
d’Aspremont, A. (2008), ‘Smooth optimization with approximate gradient’, SIAM J. Optim. 19, 11711183.
Attouch, H., Bolte, J. and Svaiter, B. F. (2013), ‘Convergence of descent methods for semi-algebraic and tame problems: Proximal algorithms, forward–backward splitting, and regularized Gauss–Seidel methods’, Math. Program. A 137, 91129.
Attouch, H., Bolte, J., Redont, P. and Soubeyran, A. (2010), ‘Proximal alternating minimization and projection methods for nonconvex problems: An approach based on the Kurdyka–Łojasiewicz inequality’, Math. Oper. Res. 35, 438457.
Attouch, H., Briceño-Arias, L. M. and Combettes, P. L. (2009/10), ‘A parallel splitting method for coupled monotone inclusions’, SIAM J. Control Optim. 48, 32463270.
Attouch, H., Buttazzo, G. and Michaille, G. (2014), Variational Analysis in Sobolev and BV Spaces: Applications to PDEs and Optimization, MOS–SIAM Series on Optimization, second edition, SIAM.
Aujol, J.-F. and Dossal, C. (2015), ‘Stability of over-relaxations for the forward–backward algorithm, application to FISTA’, SIAM J. Optim. 25, 24082433.
Auslender, A. (1976), Optimisation: Méthodes Numériques, Maîtrise de Mathématiques et Applications Fondamentales, Masson.
Auslender, A. and Teboulle, M. (2004), ‘Interior gradient and epsilon-subgradient descent methods for constrained convex minimization’, Math. Oper. Res. 29, 126.
Bauschke, H. H. and Combettes, P. L. (2011), Convex Analysis and Monotone Operator Theory in Hilbert Spaces, CMS Books in Mathematics/Ouvrages de Mathématiques de la SMC, Springer.
Bauschke, H. H., Moffat, S. M. and Wang, X. (2012), ‘Firmly nonexpansive mappings and maximally monotone operators: Correspondence and duality’, Set-Valued Var. Anal. 20, 131153.
Beck, A. (2015), ‘On the convergence of alternating minimization for convex programming with applications to iteratively reweighted least squares and decomposition schemes’, SIAM J. Optim. 25, 185209.
Beck, A. and Teboulle, M. (2003), ‘Mirror descent and nonlinear projected subgradient methods for convex optimization’, Oper. Res. Lett. 31, 167175.
Beck, A. and Teboulle, M. (2009), ‘A fast iterative shrinkage-thresholding algorithm for linear inverse problems’, SIAM J. Imaging Sci. 2, 183202.
Beck, A. and Tetruashvili, L. (2013), ‘On the convergence of block coordinate descent type methods’, SIAM J. Optim. 23, 20372060.
Becker, S. and Combettes, P. L. (2014), ‘An algorithm for splitting parallel sums of linearly composed monotone operators, with applications to signal recovery’, J. Nonlinear Convex Anal. 15, 137159.
Becker, S. and Fadili, J. (2012), A quasi-Newton proximal splitting method. In Advances in Neural Information Processing Systems 25, pp. 26272635.
Becker, S., Bobin, J. and Candès, E. (2011), ‘NESTA: A fast and accurate first-order method for sparse recovery’, SIAM J. Imaging Sci. 4, 139.
Bect, J., Blanc-Féraud, L., Aubert, G. and Chambolle, A. (2004), A l 1 -unified framework for image restoration. In Proc. 8th European Conference on Computer Vision: ECCV 2004, Part IV (Pajdla, T. and Matas, J, eds), Vol. 2034 of Lecture Notes in Computer Science, Springer, pp. 113.
Ben-Tal, A. and Nemirovski, A. (1998), ‘Robust convex optimization’, Math. Oper. Res. 23, 769805.
Ben-Tal, A. and Nemirovski, A. (2001), Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications, MPS/SIAM Series on Optimization, SIAM.
Ben-Tal, A., El Ghaoui, L. and Nemirovski, A. (2009), Robust Optimization, Princeton Series in Applied Mathematics, Princeton University Press.
Benamou, J.-D., Carlier, G., Cuturi, M., Nenna, L. and Peyré, G. (2015), ‘Iterative Bregman projections for regularized transportation problems’, SIAM J. Sci. Comput. 37, A1111A1138.
Benfenati, A. and Ruggiero, V. (2013), ‘Inexact Bregman iteration with an application to Poisson data reconstruction’, Inverse Problems 29, 065016.
Bertsekas, D. P. (2015), Convex Optimization Algorithms, Athena Scientific.
Bertsekas, D. P. and Mitter, S. K. (1973), ‘Descent numerical methods for optimization problems with nondifferentiable cost functions’, SIAM J. Control 11, 637652.
Bioucas-Dias, J. and Figueiredo, M. (2007), ‘A new TwIST: Two-step iterative shrinkage/thresholding algorithms for image restoration’, IEEE Trans. Image Process. 16, 29923004.
Blake, A. and Zisserman, A. (1987), Visual Reconstruction, MIT Press.
Blomgren, P. and Chan, T. F. (1998), ‘Color TV: total variation methods for restoration of vector-valued images’, IEEE Trans. Image Process. 7, 304309.
Bolte, J., Daniilidis, A. and Lewis, A. (2006), ‘The Łojasiewicz inequality for nonsmooth subanalytic functions with applications to subgradient dynamical systems’, SIAM J. Optim. 17, 12051223.
Bolte, J., Sabach, S. and Teboulle, M. (2014), ‘Proximal alternating linearized minimization for nonconvex and nonsmooth problems’, Math. Program. A 146, 459494.
Bonettini, S. and Ruggiero, V. (2012), ‘On the convergence of primal–dual hybrid gradient algorithms for total variation image restoration’, J. Math. Imaging Vision 44, 236253.
Bonettini, S., Benfenati, A. and Ruggiero, V. (2014), Scaling techniques for ${\it\epsilon}$-subgradient projection methods. arXiv:1407.6133
Bonettini, S., Porta, F. and Ruggiero, V. (2015), A variable metric forward–backward method with extrapolation. arXiv:1506.02900
Bonnans, J. F., Gilbert, J. C., Lemaréchal, C. and Sagastizábal, C. A. (1995), ‘A family of variable metric proximal methods’, Math. Program. A 68, 1547.
Bonnet, A. and David, G. (2001), Cracktip is a global Mumford–Shah minimizer. Astérisque, Vol. 274, Sociéte Mathématique de France.
Borwein, J. and Luke, D. (2015), Duality and convex programming. In Handbook of Mathematical Methods in Imaging (Scherzer, O., ed.), Springer, pp. 257304.
Boţ, R. I., Csetnek, E. R., Heinrich, A. and Hendrich, C. (2015), ‘On the convergence rate improvement of a primal–dual splitting algorithm for solving monotone inclusion problems’, Math. Program. A 150, 251279.
Bouchitté, G. (2006), Convex analysis and duality methods. In Encyclopedia of Mathematical Physics (Françoise, J.-P., Naber, G. L. and Tsun, T. S., eds), Academic, pp. 642652.
Boyd, S. and Vandenberghe, L. (2004), Convex Optimization, Cambridge University Press.
Boyd, S., Parikh, N., Chu, E., Peleato, B. and Eckstein, J. (2011), ‘Distributed optimization and statistical learning via the alternating direction method of multipliers’, Found. Trends Mach. Learn. 3, 1122.
Boykov, Y. and Kolmogorov, V. (2004), ‘An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision’, IEEE Trans. PAMI 26, 11241137.
Boykov, Y., Veksler, O. and Zabih, R. (2001), ‘Fast approximate energy minimization via graph cuts’, IEEE Trans. PAMI 23, 12221239.
Brakke, K. A. (1995), ‘Soap films and covering spaces’, J. Geom. Anal. 5, 445514.
Bredies, K. and Lorenz, D. A. (2008), ‘Linear convergence of iterative soft-thresholding’, J. Fourier Anal. Appl. 14, 813837.
Bredies, K. and Sun, H. (2015a), Preconditioned alternating direction method of multipliers for the minimization of quadratic plus non-smooth convex functionals. Technical report, University of Graz.
Bredies, K. and Sun, H. (2015b), ‘Preconditioned Douglas–Rachford splitting methods for convex-concave saddle-point problems’, SIAM J. Numer. Anal. 53, 421444.
Bredies, K. and Sun, H. P. (2015c), ‘Preconditioned Douglas–Rachford algorithms for TV- and TGV-regularized variational imaging problems’, J. Math. Imaging Vision 52, 317344.
Bredies, K., Kunisch, K. and Pock, T. (2010), ‘Total generalized variation’, SIAM J. Imaging Sci. 3, 492526.
Bredies, K., Lorenz, D. A. and Reiterer, S. (2015a), ‘Minimization of non-smooth, non-convex functionals by iterative thresholding’, J. Optim. Theory Appl. 165, 78112.
Bredies, K., Pock, T. and Wirth, B. (2013), ‘Convex relaxation of a class of vertex penalizing functionals’, J. Math. Imaging Vision 47, 278302.
Bredies, K., Pock, T. and Wirth, B. (2015b), ‘A convex, lower semicontinuous approximation of Euler’s elastica energy’, SIAM J. Math. Anal. 47, 566613.
Bregman, L. M. (1967), ‘A relaxation method of finding a common point of convex sets and its application to the solution of problems in convex programming’, Z̆. Vyčisl. Mat. i Mat. Fiz. 7, 620631.
Bresson, X. and Chan, T. F. (2008), ‘Fast dual minimization of the vectorial total variation norm and applications to color image processing’, Inverse Probl. Imaging 2, 455484.
Brézis, H. (1973), Opérateurs Maximaux Monotones et Semi-Groupes de Contractions dans les Espaces de Hilbert, Vol. 5 of North-Holland Mathematics Studies, North-Holland.
Brézis, H. (1983), Analyse Fonctionnelle: Théorie et Applications, Collection Mathématiques Appliquées pour la Maîtrise, Masson.
Brézis, H. and Lions, P.-L. (1978), ‘Produits infinis de résolvantes’, Israel J. Math. 29, 329345.
Briceno-Arias, L. M. and Combettes, P. L. (2011), ‘A monotone $+$ skew splitting model for composite monotone inclusions in duality’, SIAM J. Optim. 21, 12301250.
Browder, F. E. (1967), ‘Convergence theorems for sequences of nonlinear operators in Banach spaces’, Math. Z. 100, 201225.
Browder, F. E. and Petryshyn , W. V. (1966), ‘The solution by iteration of nonlinear functional equations in Banach spaces’, Bull. Am. Math. Soc. 72, 571575.
Brox, T., Bruhn, A., Papenberg, N. and Weickert, J. (2004), High accuracy optical flow estimation based on a theory for warping. In Proc. 8th European Conference on Computer Vision: ECCV 2004, Part IV (Pajdla, T. and Matas, J., eds), Vol. 2034 of Lecture Notes In Computer Science, Springer, pp. 2536.
Bruna, J. and Mallat, S. (2013), ‘Invariant scattering convolution networks’, IEEE Trans. PAMI 35, 18721886.
Bruna, J., Szlam, A. and Lecun, Y. (2014), Signal recovery from pooling representations. In 31st International Conference on Machine Learning, ICML 2014, Vol. 2, International Machine Learning Society (IMLS), pp. 15851598.
Buades, A., Coll, B. and Morel, J. M. (2005), ‘A review of image denoising algorithms, with a new one’, Multiscale Model. Simul. 4, 490530.
Buades, A., Coll, B. and Morel, J. M. (2011), ‘Non-local means denoising’, Image Processing On Line 1.
Burger, M., Sawatzky, A. and Steidl, G. (2014), First order algorithms in variational image processing. arXiv:1412.4237. To appear in Splitting Methods in Communication and Imaging, Science and Engineering (R. Glowinski, S. J. Osher and W. Yin, eds), Springer.
Burges, C. J. C. (1998), ‘A tutorial on support vector machines for pattern recognition’, Data Min. Knowl. Discov. 2, 121167.
Burke, J. V. and Qian, M. (1999), ‘A variable metric proximal point algorithm for monotone operators’, SIAM J. Control Optim. 37, 353375.
Burke, J. V. and Qian, M. (2000), ‘On the superlinear convergence of the variable metric proximal point algorithm using Broyden and BFGS matrix secant updating’, Math. Program. A 88, 157181.
Byrd, R. H., Lu, P., Nocedal, J. and Zhu, C. Y. (1995), ‘A limited memory algorithm for bound constrained optimization’, SIAM J. Sci. Comput. 16, 11901208.
Cai, J.-F., Candès, E. J. and Shen, Z. (2010), ‘A singular value thresholding algorithm for matrix completion’, SIAM J. Optim. 20, 19561982.
Candès, E., Demanet, L., Donoho, D. and Ying, L. (2006a), ‘Fast discrete curvelet transforms’, Multiscale Model. Simul. 5, 861899.
Candès, E. J., Li, X., Ma, Y. and Wright, J. (2011), ‘Robust principal component analysis?J. Assoc. Comput. Mach. 58, #11.
Candès, E. J., Romberg, J. and Tao, T. (2006b), Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inform. Theory, pp. 489509.
, Chambolle, A. (1994), Partial differential equations and image processing. In Proc. ICIP-94: 1994 IEEE International Conference on Image Processing, pp. 1620.
Chambolle, A. (1999), ‘Finite-differences discretizations of the Mumford–Shah functional’, M2AN Math. Model. Numer. Anal. 33, 261288.
Chambolle, A. (2004a), ‘An algorithm for mean curvature motion’, Interfaces Free Bound. 6, 195218.
Chambolle, A. (2004b), ‘An algorithm for total variation minimization and applications’, J. Math. Imaging Vision 20, 8997.
. Chambolle, A. (2005), Total variation minimization and a class of binary MRF models. In Energy Minimization Methods in Computer Vision and Pattern Recognition, Vol. 3757 of Lecture Notes in Computer Science, Springer, pp. 136152.
Chambolle, A. and Darbon, J. (2009), ‘On total variation minimization and surface evolution using parametric maximum flows’, Internat. J. Comput. Vis. 84, 288307.
Chambolle, A. and Darbon, J. (2012), A parametric maximum flow approach for discrete total variation regularization. In Image Processing and Analysis with Graphs: Theory and Practice (Lézoray, O. and Grady, L., eds), CRC Press, pp. 93109.
Chambolle, A. and Dossal, C. (2015), ‘On the convergence of the iterates of the fast iterative shrinkage/thresholding algorithm’, J. Optim. Theory Appl. 166, 968982.
Chambolle, A. and Lions, P.-L. (1995), Image restoration by constrained total variation minimization and variants. In Investigative and Trial Image Processing, Proc. SPIE 2567, pp. 5059.
Chambolle, A. and Lions, P.-L. (1997), ‘Image recovery via total variation minimization and related problems’, Numer. Math. 76, 167188.
Chambolle, A. and Pock, T. (2011), ‘A first-order primal–dual algorithm for convex problems with applications to imaging’, J. Math. Imaging Vision 40, 120145.
Chambolle, A. and Pock, T. (2015a), ‘On the ergodic convergence rates of a first-order primal–dual algorithm’, Math. Program. A, doi:10.1007/s10107-015-0957-3
Chambolle, A. and Pock, T. (2015b), ‘A remark on accelerated block coordinate descent for computing the proximity operators of a sum of convex functions’, SMAI J. Comput. Math. 1, 2954.
Chambolle, A., Cremers, D. and Pock, T. (2012), ‘A convex approach to minimal partitions’, SIAM J. Imaging Sci. 5, 11131158.
Chambolle, A., DeVore, R. A., Lee, N.-Y. and Lucier, B. J. (1998), ‘Nonlinear wavelet image processing: Variational problems, compression, and noise removal through wavelet shrinkage’, IEEE Trans. Image Process. 7, 319335.
Chambolle, A., Levine, S. E. and Lucier, B. J. (2011), ‘An upwind finite-difference method for total variation-based image smoothing’, SIAM J. Imaging Sci. 4, 277299.
Chan, T. F. and Esedoḡlu, S. (2005), ‘Aspects of total variation regularized $L^{1}$ function approximation’, SIAM J. Appl. Math. 65, 18171837.
Chan, T. F. and Vese, L. A. (2001), ‘Active contours without edges’, IEEE Trans. Image Process. 10, 266277.
Chan, T. F. and Vese, L. A. (2002), Active contour and segmentation models using geometric PDE’s for medical imaging. In Geometric Methods in Bio-medical Image Processing, Mathematics and Visualization, Springer, pp. 6375.
Chan, T. F., Esedoḡlu, S. and Nikolova, M. (2006), ‘Algorithms for finding global minimizers of image segmentation and denoising models’, SIAM J. Appl. Math. 66, 16321648.
Chan, T. F., Golub, G. H. and Mulet, P. (1999), ‘A nonlinear primal–dual method for total variation-based image restoration’, SIAM J. Sci. Comput. 20, 19641977.
Chartrand, R. and Wohlberg, B. (2013), A nonconvex ADMM algorithm for group sparsity with sparse groups. In Proc. 2013 IEEE International Conference on Acoustics, Speech and Signal Processing: ICASSP, IEEE, pp. 60096013.
Chen, G. and Teboulle, M. (1993), ‘Convergence analysis of a proximal-like minimization algorithm using Bregman functions’, SIAM J. Optim. 3, 538543.
Chen, G. and Teboulle, M. (1994), ‘A proximal-based decomposition method for convex minimization problems’, Math. Program. A 64, 81101.
Chen, S. and Donoho, D. (1994), Basis pursuit. In 28th Asilomar Conference on Signals, Systems, and Computers, IEEE, pp. 4144.
Chen, S. S., Donoho, D. L. and Saunders, M. A. (1998), ‘Atomic decomposition by basis pursuit’, SIAM J. Sci. Comput. 20, 3361.
Chen, Y., Lan, G. and Ouyang, Y. (2014a), ‘Optimal primal–dual methods for a class of saddle point problems’, SIAM J. Optim. 24, 17791814.
Chen, Y., Ranftl, R. and Pock, T. (2014b), ‘Insights into analysis operator learning: From patch-based sparse models to higher order MRFs’, IEEE Trans. Image Process. 23, 10601072.
Chouzenoux, E., Pesquet, J.-C. and Repetti, A. (2014), ‘Variable metric forward–backward algorithm for minimizing the sum of a differentiable function and a convex function’, J. Optim. Theory Appl. 162, 107132.
Chouzenoux, E., Pesquet, J.-C. and Repetti, A. (2016), ‘A block coordinate variable metric forward–backward algorithm’, J. Global Optim., doi:10.1007/s10898-016-0405-9
Ciresan, D. C., Meier, U. and Schmidhuber, J. (2012), Multi-column deep neural networks for image classification. In Proc. 25th IEEE Conference on Computer Vision and Pattern Recognition: CVPR 2012, pp. 36423649.
Citti, G. and Sarti, A. (2006), ‘A cortical based model of perceptual completion in the roto-translation space’, J. Math. Imaging Vision 24, 307326.
Combettes, P. L. (2004), ‘Solving monotone inclusions via compositions of nonexpansive averaged operators’, Optimization 53, 475504.
Combettes, P. L. and Pesquet, J.-C. (2011), Proximal splitting methods in signal processing. In Fixed-Point Algorithms for Inverse Problems in Science and Engineering, Vol. 49 of Springer Optimization and its Applications, Springer, pp. 185212.
Combettes, P. L. and , B. C. (2014), ‘Variable metric forward–backward splitting with applications to monotone inclusions in duality’, Optimization 63, 12891318.
Combettes, P. L. and Wajs, V. R. (2005), ‘Signal recovery by proximal forward–backward splitting’, Multiscale Model. Simul. 4, 11681200.
Condat, L. (2013a), ‘A direct algorithm for 1D total variation denoising’, IEEE Signal Proc. Letters 20, 10541057.
Condat, L. (2013b), ‘A primal–dual splitting method for convex optimization involving Lipschitzian, proximable and linear composite terms’, J. Optim. Theory Appl. 158, 460479.
Cremers, D. and Strekalovskiy, E. (2013), ‘Total cyclic variation and generalizations’, J. Math. Imaging Vision 47, 258277.
Dabov, K., Foi, A., Katkovnik, V. and Egiazarian, K. (2007), ‘Image denoising by sparse 3-D transform-domain collaborative filtering’, IEEE Trans. Image Process. 16, 20802095.
Dal Maso, G., Mora, M. G. and Morini, M. (2000), ‘Local calibrations for minimizers of the Mumford–Shah functional with rectilinear discontinuity sets’, J. Math. Pures Appl. (9) 79, 141162.
Danielyan, A., Katkovnik, V. and Egiazarian, K. (2012), ‘BM3D frames and variational image deblurring’, IEEE Trans. Image Process. 21, 17151728.
Darbon, J. and Sigelle, M. (2004), Exact optimization of discrete constrained total variation minimization problems. In Combinatorial Image Analysis, Vol. 3322 of Lecture Notes in Computer Science, Springer, pp. 548557.
Darbon, J. and Sigelle, M. (2006a), ‘Image restoration with discrete constrained total variation I: Fast and exact optimization’, J. Math. Imaging Vision 26, 261276.
Darbon, J. and Sigelle, M. (2006b), ‘Image restoration with discrete constrained total variation II: Levelable functions, convex priors and non-convex cases’, J. Math. Imaging Vision 26, 277291.
Daubechies, I., Defrise, M. and De Mol, C. (2004), ‘An iterative thresholding algorithm for linear inverse problems with a sparsity constraint’, Comm. Pure Appl. Math. 57, 14131457.
Davies, P. L. and Kovac, A. (2001), ‘Local extremes, runs, strings and multiresolution’, Ann. Statist. 29, 165.
Davis, D. (2015), ‘Convergence rate analysis of primal–dual splitting schemes’, SIAM J. Optim. 25, 19121943.
Davis, D. and Yin, W. (2014a), Convergence rate analysis of several splitting schemes. arXiv:1406.4834
Davis, D. and Yin, W. (2014b), Faster convergence rates of relaxed Peaceman–Rachford and ADMM under regularity assumptions. arXiv:1407.5210
Davis, D. and Yin, W. (2015) A three-operator splitting scheme and its optimization applications. CAM Report 15-13, UCLA. arXiv:1504.01032
Deng, W. and Yin, W. (2016), ‘On the global and linear convergence of the generalized alternating direction method of multipliers’, J. Sci. Comput. 66, 889916.
DeVore, R. A. (1998), Nonlinear approximation. In Acta Numerica, Vol. 7, Cambridge University Press, pp. 51150.
Donoho, D. L. (1995), ‘De-noising by soft-thresholding’, IEEE Trans. Inform. Theory 41, 613627.
Donoho, D. L. (2006), ‘Compressed sensing’, IEEE Trans. Inform. Theory 52, 12891306.
Douglas, J. and Rachford, H. H. (1956), ‘On the numerical solution of heat conduction problems in two and three space variables’, Trans. Amer. Math. Soc. 82, 421439.
Drori, Y., Sabach, S. and Teboulle, M. (2015), ‘A simple algorithm for a class of nonsmooth convex-concave saddle-point problems’, Oper. Res. Lett. 43, 209214.
Duan, K.-B. and Keerthi, S. S. (2005), Which is the best multiclass SVM method? An empirical study. In Multiple Classifier Systems: Proc. 6th International Workshop, MCS 2005 (Oza, N. C., Polikar, R., Kittler, J. and Roli, F., eds), Vol. 3541 of Lecture Notes in Computer Science, Springer, pp. 278285.
Duchi, J., Shalev-Shwartz, S., Singer, Y. and Chandra, T. (2008), Efficient projections onto the 1-ball for learning in high dimensions. In Proc. 25th International Conference on Machine Learning: ICML ’08, ACM, pp. 272279.
Dupé, F.-X., Fadili, M. J. and Starck, J.-L. (2012), ‘Deconvolution under Poisson noise using exact data fidelity and synthesis or analysis sparsity priors’, Statist. Methodol. 9, 418.
Duran, J., Moeller, M., Sbert, C. and Cremers, D. (2016a), ‘On the implementation of collaborative TV regularization: Application to cartoon+texture decomposition’, Image Processing On Line 6, 2774.
Duran, J., Möller, M., Sbert, C. and Cremers, D. (2016b), ‘Collaborative total variation: A general framework for vectorial TV models’, SIAM J. Imaging Sci. 9, 116151.
Dykstra, R. L. (1983), ‘An algorithm for restricted least squares regression’, J. Amer. Statist. Assoc. 78(384), 837842.
Easley, G., Labate, D. and Lim, W.-Q. (2008), ‘Sparse directional image representations using the discrete shearlet transform’, Appl. Comput. Harmon. Anal. 25, 2546.
Eckstein, J. (1989) Splitting methods for monotone operators with applications to parallel optimization. PhD thesis, Massachusetts Institute of Technology.
Eckstein, J. (1993), ‘Nonlinear proximal point algorithms using Bregman functions, with applications to convex programming’, Math. Oper. Res. 18, 202226.
Eckstein, J. and Bertsekas, D. P. (1992), ‘On the Douglas–Rachford splitting method and the proximal point algorithm for maximal monotone operators’, Math. Program. A 55, 293318.
Ekeland, I. and Témam, R. (1999), Convex Analysis and Variational Problems (translated from French), Vol. 28 of Classics in Applied Mathematics, SIAM.
Esser, E. (2009) Applications of Lagrangian-based alternating direction methods and connections to split Bregman. CAM Report 09-31, UCLA.
Esser, E., Zhang, X. and Chan, T. F. (2010), ‘A general framework for a class of first order primal–dual algorithms for convex optimization in imaging science’, SIAM J. Imaging Sci. 3, 10151046.
Evans, L. C. and Gariepy, R. F. (1992), Measure Theory and Fine Properties of Functions, CRC Press.
Federer, H. (1969), Geometric Measure Theory, Springer.
Fercoq, O. and Bianchi, P. (2015), A coordinate descent primal–dual algorithm with large step size and possibly non separable functions. arXiv:1508.04625
Fercoq, O. and Richtárik, P. (2013), Smooth minimization of nonsmooth functions with parallel coordinate descent methods. arXiv:1309.5885
Fercoq, O. and Richtárik, P. (2015), ‘Accelerated, parallel and proximal coordinate descent’, SIAM J. Optim. 25, 19972023.
Ferradans, S., Papadakis, N., Peyré, G. and Aujol, J.-F. (2014), ‘Regularized discrete optimal transport’, SIAM J. Imaging Sci. 7, 18531882.
Fortin, M. and Glowinski, R. (1982), Méthodes de Lagrangien Augmenté: Applications à la Résolution Numérique de Problèmes aux Limites, Vol. 9 of Méthodes Mathématiques de l’Informatique, Gauthier-Villars.
Fu, X. L., He, B. S., Wang, X. F. and Yuan, X. M. (2014) ‘Block-wise alternating direction method of multipliers with Gaussian back substitution for multiple-block convex programming’.
Fukushima, M. and Mine, H. (1981), ‘A generalized proximal point algorithm for certain nonconvex minimization problems’, Internat. J. Systems Sci. 12, 9891000.
Gabay, D. (1983), Applications of the method of multipliers to variational inequalities. In Chapter IX of Augmented Lagrangian Methods: Applications to the Solution of Boundary Value Problems (Fortin, M. and Glowinski, R., eds), North-Holland, pp. 299340.
Gabay, D. and Mercier, B. (1976), ‘A dual algorithm for the solution of nonlinear variational problems via finite element approximation’, Comput. Math. Appl. 2, 1740.
Gallo, G., Grigoriadis, M. D. and Tarjan, R. E. (1989), ‘A fast parametric maximum flow algorithm and applications’, SIAM J. Comput. 18, 3055.
Geman, D. and Reynolds, G. (1992), ‘Constrained restoration and the recovery of discontinuities’, IEEE Trans. PAMI 14, 367383.
Geman, S. and Geman, D. (1984), ‘Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images’, IEEE Trans. PAMI 6, 721741.
Getreuer, P. (2012), ‘Total variation deconvolution using split Bregman’, Image Processing On Line 2, 158174.
Gilboa, G., Darbon, J., Osher, S. and Chan, T. 2006, ‘Nonlocal convex functionals for image regularization. CAM Report 06-57, UCLA.
Giusti, E. (1984), Minimal Surfaces and Functions of Bounded Variation, Birkhäuser.
Glowinski, R. and Le Tallec, P. (1989), Augmented Lagrangian and Operator-Splitting Methods in Nonlinear Mechanics, Vol. 9 of SIAM Studies in Applied Mathematics, SIAM.
Glowinski, R. and Marroco, A. (1975), ‘Sur l’approximation, par éléments finis d’ordre un, et la résolution, par pénalisation-dualité, d’une classe de problèmes de Dirichlet non linéaires’, Rev. Française Automat. Informat. Recherche Opérationnelle Sér. Rouge Anal. Numér. 9(R-2), 4176.
Goldfarb, D. and Ma, S. (2012), ‘Fast multiple-splitting algorithms for convex optimization’, SIAM J. Optim. 22, 533556.
Goldluecke, B., Strekalovskiy, E. and Cremers, D. (2012), ‘The natural vectorial total variation which arises from geometric measure theory’, SIAM J. Imaging Sci. 5, 537564.
Goldluecke, B., Strekalovskiy, E. and Cremers, D. (2013), ‘Tight convex relaxations for vector-valued labeling’, SIAM J. Imaging Sci. 6, 16261664.
Goldstein, A. A. (1964), ‘Convex programming in Hilbert space’, Bull. Amer. Math. Soc. 70, 709710.
Goldstein, T. and Osher, S. (2009), ‘The split Bregman method for $L^{1}$-regularized problems’, SIAM J. Imaging Sci. 2, 323343.
Goldstein, T., Li, M., Yuan, X., Esser, E. and Baraniuk, R. (2015), Adaptive primal–dual hybrid gradient methods for saddle-point problems. arXiv:1305.0546v2
Goldstein, T., O’Donoghue, B., Setzer, S. and Baraniuk, R. (2014), ‘Fast alternating direction optimization methods’, SIAM J. Imaging Sci. 7, 15881623.
Grasmair, M. (2010), ‘Non-convex sparse regularisation’, J. Math. Anal. Appl. 365, 1928.
Grasmair, M., Haltmeier, M. and Scherzer, O. (2011), ‘Necessary and sufficient conditions for linear convergence of $\ell ^{1}$-regularization’, Comm. Pure Appl. Math. 64, 161182.
Grippo, L. and Sciandrone, M. (2000), ‘On the convergence of the block nonlinear Gauss–Seidel method under convex constraints’, Oper. Res. Lett. 26, 127136.
Güler, O. (1991), ‘On the convergence of the proximal point algorithm for convex minimization’, SIAM J. Control Optim. 29, 403419.
Güler, O. (1992), ‘New proximal point algorithms for convex minimization’, SIAM J. Optim. 2, 649664.
Guo, K. and Labate, D. (2007), ‘Optimally sparse multidimensional representation using shearlets’, SIAM J. Math. Analysis 39, 298318.
Guo, K., Kutyniok, G. and Labate, D. (2006), Sparse multidimensional representations using anisotropic dilation and shear operators. In Wavelets and Splines: Athens 2005, Nashboro Press, pp. 189201.
Hashimoto, W. and Kurata, K. (2000), ‘Properties of basis functions generated by shift invariant sparse representations of natural images’, Biological Cybernetics 83, 111118.
Hawe, S., Kleinsteuber, M. and Diepold, K. (2013), ‘Analysis operator learning and its application to image reconstruction’, IEEE Trans. Image Process. 22, 21382150.
He, B. and Yuan, X. (2015a), ‘Block-wise alternating direction method of multipliers for multiple-block convex programming and beyond’, SMAI J. Comput. Math. 1, 145174.
He, B. and Yuan, X. (2015b), ‘On non-ergodic convergence rate of Douglas–Rachford alternating direction method of multipliers’, Numer. Math. 130, 567577.
He, B. and Yuan, X. (2015c), ‘On the convergence rate of Douglas–Rachford operator splitting method’, Math. Program. A 153, 715722.
He, B., You, Y. and Yuan, X. (2014), ‘On the convergence of primal–dual hybrid gradient algorithm’, SIAM J. Imaging Sci. 7, 25262537.
Hestenes, M. R. (1969), ‘Multiplier and gradient methods’, J. Optim. Theory Appl. 4, 303320.
Hochbaum, D. S. (2001), ‘An efficient algorithm for image segmentation, Markov random fields and related problems’, J. Assoc. Comput. Mach. 48, 686701.
Hohage, T. and Homann, C. (2014) A generalization of the Chambolle–Pock algorithm to Banach spaces with applications to inverse problems. arXiv:1412.0126
Hong, M., Luo, Z.-Q. and Razaviyayn, M. (2015), Convergence analysis of alternating direction method of multipliers for a family of nonconvex problems. In Proc. 2015 IEEE International Conference on Acoustics, Speech and Signal Processing: ICASSP, pp. 38363840.
Horn, B. K. P. and Schunck, B. G. (1981), ‘Determining optical flow’, Artif. Intell. 17, 185203.
Hubel, D. and Wiesel, T. (1959), ‘Receptive fields of single neurones in the cat’s striate cortex’, J. Physiology 148, 574591.
Ito, K. and Kunisch, K. (1990), ‘The augmented Lagrangian method for equality and inequality constraints in Hilbert spaces’, Math. Program. 46, 341360.
Johnson, N. A. (2013), ‘A dynamic programming algorithm for the fused Lasso and $l_{0}$-segmentation’, J. Comput. Graph. Statist 22, 246260.
Kanizsa, G. (1979), Organization in Vision, Praeger.
Kindermann, S., Osher, S. and Jones, P. W. (2005), ‘Deblurring and denoising of images by nonlocal functionals’, Multiscale Model. Simul. 4, 10911115.
Kiwiel, K. C. (1997), ‘Proximal minimization methods with generalized Bregman functions’, SIAM J. Control Optim. 35, 11421168.
Knoll, F., Bredies, K., Pock, T. and Stollberger, R. (2011), ‘Second order total generalized variation (TGV) for MRI’, Magnetic Resonance in Medicine 65, 480491.
Kolmogorov, V., Pock, T. and Rolinek, M. (2016), ‘Total variation on a tree’, SIAM J. Imaging Sci., accepted.
Korpelevich, G. M. (1976), ‘An extragradient method for finding saddle points and other problems’, Ekonom. i Mat. Metody 12, 747756.
Korpelevich, G. M. (1983), ‘Extrapolational gradient methods and their connection with modified Lagrangians’, Ehkon. Mat. Metody 19, 694703.
Krasnosel’skiĭ, M. A. (1955), ‘Two remarks on the method of successive approximations’, Uspekhi Mat. Nauk (N.S.) 10, 123127.
Krizhevsky, A., Sutskever, I. and Hinton, G. E. (2012), Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems 25: Proc. NIPS, pp. 11061114.
Kutyniok, G. and Lim, W.-Q. (2011), ‘Compactly supported shearlets are optimally sparse’, J. Approx. Theory 163, 15641589.
Lawlor, G. and Morgan, F. (1994), ‘Paired calibrations applied to soap films, immiscible fluids, and surfaces or networks minimizing other norms’, Pacific J. Math. 166, 5583.
Lebrun, M., Buades, A. and Morel, J. M. (2013), ‘A nonlocal Bayesian image denoising algorithm’, SIAM J. Imaging Sci. 6, 16651688.
LeCun, Y., Boser, B. E., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W. E. and Jackel, L. D. (1989), Handwritten digit recognition with a back-propagation network. In Advances in Neural Information Processing Systems 2: Proc. NIPS 1989, pp. 396404.
LeCun, Y., Bottou, L., Bengio, Y. and Haffner, P. (1998a), ‘Gradient-based learning applied to document recognition’, Proc. IEEE 86, 22782324.
LeCun, Y., Bottou, L., Orr, G. and Muller, K. (1998b), Efficient BackProp. In Neural Networks: Tricks of the Trade (Orr, G. and Muller, K., eds), Springer.
Lee, D. D. and Seung, H. S. (1999), ‘Learning the parts of objects by nonnegative matrix factorization’, Nature 401, 788791.
Lee, H., Battle, A., Raina, R. and Ng, A. Y. (2007), Efficient sparse coding algorithms. In Advances in Neural Information Processing Systems 19 (Schölkopf, B., Platt, J. and Hoffman, T., eds), MIT Press, pp. 801808.
Lellmann, J. and Schnörr, C. (2011), ‘Continuous multiclass labeling approaches and algorithms’, SIAM J. Imaging Sci. 4, 10491096.
Lellmann, J., Lenzen, F. and Schnörr, C. (2013), ‘Optimality bounds for a variational relaxation of the image partitioning problem’, J. Math. Imaging Vision 47, 239257.
Lessard, L., Recht, B. and Packard, A. (2016), ‘Analysis and design of optimization algorithms via integral quadratic constraints’, SIAM J. Optim. 26, 5795.
Levin, A., Weiss, Y., Durand, F. and Freeman, W. T. (2011), Efficient marginal likelihood optimization in blind deconvolution. In Proc. 24th IEEE Conference on Computer Vision and Pattern Recognition: CVPR 2011, pp. 26572664.
Liang, J., Fadili, J. and Peyré, G. (2014), Local linear convergence of forward–backward under partial smoothness. In Advances in Neural Information Processing Systems 27: Proc. NIPS 2014, pp. 19701978.
Liang, J., Fadili, J. and Peyré, G. (2015), ‘Convergence rates with inexact nonexpansive operators’, Math. Program. A, doi:10.1007/s10107-015-0964-4
Lin, Q., Lu, Z. and Xiao, L. (2015), ‘An accelerated proximal coordinate gradient method and its application to regularized empirical risk minimization’, SIAM J. Optim. 25, 22442273.
Lions, P. L. and Mercier, B. (1979), ‘Splitting algorithms for the sum of two nonlinear operators’, SIAM J. Numer. Anal. 16, 964979.
Lucas, B. D. and Kanade, T. (1981), An iterative image registration technique with an application to stereo vision. In Proc. 7th International Joint Conference on Artificial Intelligence: IJCAI ’81, pp. 674679.
Magnússon, S., Chathuranga Weeraddana, P., Rabbat, M. G. and Fischione, C. (2014), On the convergence of alternating direction Lagrangian methods for nonconvex structured optimization problems. arXiv:1409.8033
Mahendran, A. and Vedaldi, A. (2015), Understanding deep image representations by inverting them. In Proc. IEEE Conference on Computer Vision and Pattern Recognition: CVPR 2015, pp. 51885196.
Mairal, J., Bach, F., Ponce, J. and Sapiro, G. (2009a), Online dictionary learning for sparse coding. In Proc. 26th Annual International Conference on Machine Learning: ICML ’09, ACM, pp. 689696.
Mairal, J., Ponce, J., Sapiro, G., Zisserman, A. and Bach, F. R. (2009b), Supervised dictionary learning. In Advances in Neural Information Processing Systems 21: Proc. NIPS 2009 (Koller, D., Schuurmans, D., Bengio, Y. and Bottou, L., eds), Curran Associates, pp. 10331040.
Mallat, S. and Yu, G. (2010), ‘Super-resolution with sparse mixing estimators’, IEEE Trans. Image Process. 19, 28892900.
Mallat, S. and Zhang, Z. (1993), ‘Matching pursuits with time-frequency dictionaries’, Trans. Signal Process. 41, 33973415.
Mann, W. R. (1953), ‘Mean value methods in iteration’, Proc. Amer. Math. Soc. 4, 506510.
Martinet, B. (1970), ‘Brève communication. régularisation d’inéquations variationnelles par approximations successives’, ESAIM: Mathematical Modelling and Numerical Analysis / Modélisation Mathématique et Analyse Numérique 4(R3), 154158.
Masnou, S. and Morel, J.-M. (2006), ‘On a variational theory of image amodal completion’, Rend. Sem. Mat. Univ. Padova 116, 211252.
Mine, H. and Fukushima, M. (1981), ‘A minimization method for the sum of a convex function and a continuously differentiable function’, J. Optim. Theory Appl. 33, 923.
Minty, G. J. (1962), ‘Monotone (nonlinear) operators in Hilbert space’, Duke Math. J. 29, 341346.
Möllenhoff, T., Strekalovskiy, E., Moeller, M. and Cremers, D. (2015), ‘The primal–dual hybrid gradient method for semiconvex splittings’, SIAM J. Imaging Sci. 8, 827857.
Mora, M. G. and Morini, M. (2001), ‘Local calibrations for minimizers of the Mumford–Shah functional with a regular discontinuity set’, Ann. Inst. H. Poincaré Anal. Non Linéaire 18, 403436.
Morales, J. L. and Nocedal, J. (2011), ‘Remark on “Algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound constrained optimization” [mr1671706]’, ACM Trans. Math. Software 38, 7.
Moreau, J. J. (1965), ‘Proximité et dualité dans un espace Hilbertien’, Bull. Soc. Math. France 93, 273299.
Moudafi, A. and Oliny, M. (2003), ‘Convergence of a splitting inertial proximal method for monotone operators’, J. Comput. Appl. Math. 155, 447454.
Mumford, D. and Shah, J. (1989), ‘Optimal approximation by piecewise smooth functions and associated variational problems’, Comm. Pure Appl. Math. 42, 577685.
Nam, S., Davies, M., Elad, M. and Gribonval, R. (2013), ‘The cosparse analysis model and algorithms’, Appl. Comput. Harmonic Anal. 34, 3056.
Nemirovski, A. S. (2004), ‘Prox-method with rate of convergence $O(1/t)$ for variational inequalities with Lipschitz continuous monotone operators and smooth convex-concave saddle point problems’, SIAM J. Optim. 15, 229251.
Nemirovski, A. S. and Yudin, D. (1983), Problem Complexity and Method Efficiency in Optimization (translated from Russian), Wiley-Interscience Series in Discrete Mathematics, Wiley.
Nesterov, Y. (1983), ‘A method for solving the convex programming problem with convergence rate $O(1/k^{2})$’, Dokl. Akad. Nauk SSSR 269, 543547.
Nesterov, Y. (2004), Introductory Lectures on Convex Optimization: A Basic Course, Vol. 87 of Applied Optimization, Kluwer Academic.
Nesterov, Y. (2005), ‘Smooth minimization of non-smooth functions’, Math. Program. 103, 127152.
Nesterov, Y. (2012), ‘Efficiency of coordinate descent methods on huge-scale optimization problems’, SIAM J. Optim. 22, 341362.
Nesterov, Y. (2013), ‘Gradient methods for minimizing composite functions’, Math. Program. B 140, 125161.
Nesterov, Y. (2015), ‘Universal gradient methods for convex optimization problems’, Math. Program. A 152, 381404.
Nikolova, M. (2004), ‘A variational approach to remove outliers and impulse noise’, J. Math. Image Vis. 20, 99120.
Nishihara, R., Lessard, L., Recht, B., Packard, A. and Jordan, M. I. (2015), A general analysis of the convergence of ADMM. In Proc. 32nd International Conference on Machine Learning, Vol. 37 of JMLR Workshop and Conference Proceedings.
Nitzberg, M., Mumford, D. and Shiota, T. (1993), Filtering, Segmentation and Depth, Vol. 662 of Lecture Notes in Computer Science, Springer.
Nocedal, J. and Wright, S. J. (2006), Numerical Optimization, Springer Series in Operations Research and Financial Engineering, second edition, Springer.
Ochs, P., Brox, T. and Pock, T. (2015), ‘iPiasco: Inertial proximal algorithm for strongly convex optimization’, J. Math. Imaging Vision 53, 171181.
Ochs, P., Chen, Y., Brox, T. and Pock, T. (2014), ‘iPiano: inertial proximal algorithm for nonconvex optimization’, SIAM J. Imaging Sci. 7, 13881419.
O’Connor, D. and Vandenberghe, L. (2014), ‘Primal–dual decomposition by operator splitting and applications to image deblurring’, SIAM J. Imaging Sci. 7, 17241754.
O’Donoghue, B. and Candès, E. (2015), ‘Adaptive restart for accelerated gradient schemes’, Found. Comput. Math. 15, 715732.
Olshausen, B. A. and Field, D. J. (1997), ‘Sparse coding with an overcomplete basis set: A strategy employed by V1?Vision Research 37, 33113325.
Opial, Z. (1967), ‘Weak convergence of the sequence of successive approximations for nonexpansive mappings’, Bull. Amer. Math. Soc. 73, 591597.
Ouyang, Y., Chen, Y., Lan, G. and Pasiliao, E. Jr (2015), ‘An accelerated linearized alternating direction method of multipliers’, SIAM J. Imaging Sci. 8, 644681.
Paatero, P. and Tapper, U. (1994), ‘Positive matrix factorization: A nonnegative factor model with optimal utilization of error estimates of data values’, Environmetrics 5, 111126.
Parikh, N. and Boyd, S. (2014), ‘Proximal algorithms’, Found. Trends Optim. 1, 127239.
Passty, G. B. (1979), ‘Ergodic convergence to a zero of the sum of monotone operators in Hilbert space’, J. Math. Anal. Appl. 72, 383390.
Patrinos, P., Stella, L. and Bemporad, A. (2014), Douglas–Rachford splitting: Complexity estimates and accelerated variants. In Proc. 2014 IEEE 53rd Annual Conference on Decision and Control: CDC, pp. 42344239.
Peyré, G., Bougleux, S. and Cohen, L. (2008), Non-local regularization of inverse problems. In Proc. 10th European Conference on Computer Vision: ECCV 2008, Vol. 5304 of Lecture Notes in Computer Science, Springer, pp. 5768.
Peyré, G., Fadili, J. and Starck, J.-L. (2010), ‘Learning the morphological diversity’, SIAM J. Imaging Sci. 3, 646669.
Picard, J. C. and Ratliff, H. D. (1975), ‘Minimum cuts and related problems’, Networks 5, 357370.
Pock, T. and Chambolle, A. (2011), Diagonal preconditioning for first order primal–dual algorithms. In Proc. IEEE International Conference on Computer Vision: ICCV 2011, IEEE, pp. 17621769.
Pock, T. and Sabach, S. (2016) Inertial proximal alternating linearized minimization (iPALM) for nonconvex and nonsmooth problems. Technical report.
Pock, T., Cremers, D., Bischof, H. and Chambolle, A. (2009), An algorithm for minimizing the Mumford–Shah functional. In Proc. IEEE 12th International Conference on Computer Vision: ICCV 2009, IEEE, pp. 11331140.
Pock, T., Cremers, D., Bischof, H. and Chambolle, A. (2010), ‘Global solutions of variational models with convex regularization’, SIAM J. Imaging Sci. 3, 11221145.
Pock, T., Schoenemann, T., Graber, G., Bischof, H. and Cremers, D. (2008), A convex formulation of continuous multi-label problems. In Proc. 10th European Conference on Computer Vision: ECCV 2008, Vol. 5304 of Lecture Notes in Computer Science, Springer, pp. 792805.
Polyak, B. T. (1987), Introduction to Optimization (translated from Russian),, Translations Series in Mathematics and Engineering, Optimization Software.
Popov, L. D. (1981), A modification of the Arrow–Hurwitz method of search for saddle points with an adaptive procedure for determining the iteration step. In Classification and Optimization in Control Problems, Akad. Nauk SSSR Ural. Nauchn. Tsentr, Sverdlovsk, pp. 5256.
Powell, M. J. D. (1969), A method for nonlinear constraints in minimization problems. In Optimization: Keele 1968, Academic, pp. 283298.
Protter, M., Yavneh, I. and Elad, M. (2010), ‘Closed-form MMSE estimation for signal denoising under sparse representation modelling over a unitary dictionary’, IEEE Trans. Signal Process. 58, 34713484.
Pustelnik, N., Chaux, C. and Pesquet, J.-C. (2011), ‘Parallel proximal algorithm for image restoration using hybrid regularization’, IEEE Trans. Image Process. 20, 24502462.
Raguet, H., Fadili, J. and Peyré, G. (2013), ‘A generalized forward–backward splitting’, SIAM J. Imaging Sci. 6, 11991226.
Rockafellar, R. T. (1976), ‘Monotone operators and the proximal point algorithm’, SIAM J. Control Optim. 14, 877898.
Rockafellar, R. T. (1997), Convex Analysis, Princeton Landmarks in Mathematics, Princeton University Press.
Rother, C., Kolmogorov, V. and Blake, A. (2004), ‘“GrabCut”: Interactive foreground extraction using iterated graph cuts’, ACM Trans. Graph. 23, 309314.
Rudin, L., Osher, S. J. and Fatemi, E. (1992), ‘Nonlinear total variation based noise removal algorithms’, Physica D 60, 259268.
Salzo, S. and Villa, S. (2012), ‘Inexact and accelerated proximal point algorithms’, J. Convex Anal. 19, 11671192.
Sapiro, G. and Ringach, D. L. (1996), ‘Anisotropic diffusion of multivalued images with applications to color filtering’, IEEE Trans. Image Process. 5, 15821586.
Schaefer, H. (1957), ‘Über die Methode sukzessiver Approximationen’, Jber. Deutsch. Math. Verein. 59, 131140.
Schmidt, M., Roux, N. L. and Bach, F. R. (2011), Convergence rates of inexact proximal-gradient methods for convex optimization. In Advances in Neural Information Processing Systems 24: Proc. NIPS 2011 (Shawe-Taylor, J., Zemel, R., Bartlett, P., Pereira, F. and Weinberger, K., eds), Curran Associates, pp. 14581466.
Schoenemann, T. and Cremers, D. (2007), Globally optimal image segmentation with an elastic shape prior. In Proc. 11th IEEE International Conference on Computer Vision: ICCV 2007, IEEE, pp. 16.
Schoenemann, T., Masnou, S. and Cremers, D. (2011), ‘The elastic ratio: Introducing curvature into ratio-based image segmentation’, IEEE Trans. Image Process. 20, 25652581.
Setzer, S. (2011), ‘Operator splittings, Bregman methods and frame shrinkage in image processing’, Internat. J. Comput. Vis. 92, 265280.
Shefi, R. and Teboulle, M. (2014), ‘Rate of convergence analysis of decomposition methods based on the proximal method of multipliers for convex minimization’, SIAM J. Optim. 24, 269297.
Sidky, E. Y., Kao, C.-M. and Pan, X. (2006), ‘Accurate image reconstruction from few-views and limited-angle data in divergent-beam CT’, 14, 119–139.
Simonyan, K. and Zisserman, A. (2015), Very deep convolutional networks for large-scale image recognition. In Proc. International Conference on Learning Representations,, arXiv:1409.1556
Starck, J.-L., Murtagh, F. and Fadili, J. M. (2010), Sparse Image and Signal Processing: Wavelets, Curvelets, Morphological Diversity, Cambridge University Press.
Steidl, G. and Teuber, T. (2010), ‘Removing multiplicative noise by Douglas–Rachford splitting methods’, J. Math. Imaging Vision 36, 168184.
Stella, L., Themelis, A. and Patrinos, P. (2016) Forward–backward quasi-Newton methods for nonsmooth optimization problems. arXiv:1604.08096v1
Strang, G. (1983), ‘Maximal flow through a domain’, Math. Program. 26, 123143.
Strang, G. (2010), ‘Maximum flows and minimum cuts in the plane’, J. Global Optim. 47, 527535.
Strekalovskiy, E., Chambolle, A. and Cremers, D. (2014), ‘Convex relaxation of vectorial problems with coupled regularization’, SIAM J. Imaging Sci. 7, 294336.
Tan, P. (2016) Acceleration of saddle-point methods in smooth cases. In preparation.
Tao, S., Boley, D. and Zhang, S. (2015) , Local linear convergence of ISTA and FISTA on the LASSO problem. arXiv:1501.02888
Teboulle, M. (1992), ‘Entropic proximal mappings with applications to nonlinear programming’, Math. Oper. Res. 17, 670690.
Tibshirani, R. (1996), ‘Regression shrinkage and selection via the lasso’, J. Royal Statist. Soc. B 58, 267288.
Tseng, P. (1991), ‘Applications of a splitting algorithm to decomposition in convex programming and variational inequalities’, SIAM J. Control Optim. 29, 119138.
Tseng, P. (2000), ‘A modified forward–backward splitting method for maximal monotone mappings’, SIAM J. Control Optim. 38, 431446.
Tseng, P. (2001), ‘Convergence of a block coordinate descent method for nondifferentiable minimization’, J. Optim. Theory Appl. 109, 475494.
Tseng, P. (2008) On accelerated proximal gradient methods for convex–concave optimization.∼b97058/tseng/papers/apgm.pdf
Tseng, P. and Yun, S. (2009), ‘Block-coordinate gradient descent method for linearly constrained nonsmooth separable optimization’, J. Optim. Theory Appl. 140, 513535.
Valkonen, T. (2014), ‘A primal–dual hybrid gradient method for nonlinear operators with applications to MRI’, Inverse Problems 30, 055012.
Valkonen, T. and Pock, T. (2015), Acceleration of the PDHGM on strongly convex subspaces. arXiv:1511.06566
Vapnik, V. N. (2000), The Nature of Statistical Learning Theory, Statistics for Engineering and Information Science, second edition, Springer.
Varga, R. S. (1962), Matrix Iterative Analysis, Prentice Hall.
Villa, S., Salzo, S., Baldassarre, L. and Verri, A. (2013), ‘Accelerated and inexact forward–backward algorithms’, SIAM J. Optim. 23, 16071633.
, B. C. (2013a), ‘A splitting algorithm for dual monotone inclusions involving cocoercive operators’, Adv. Comput. Math. 38, 667681.
, B. C. (2013b), ‘A variable metric extension of the forward–backward–forward algorithm for monotone operators’, Numer. Funct. Anal. Optim. 34, 10501065.
Wang, Y., Yin, W. and Zeng, J. (2015), Global convergence of ADMM in nonconvex nonsmooth optimization. arXiv:1511.06324
Yamagishi, M. and Yamada, I. (2011), ‘Over-relaxation of the fast iterative shrinkage-thresholding algorithm with variable stepsize’, Inverse Problems 27, 105008.
Yanez, F. and Bach, F. (2014), Primal–dual algorithms for non-negative matrix factorization with the Kullback–Leibler divergence. arXiv:1412.1788
Yin, W. and Osher, S. (2013), ‘Error forgetting of Bregman iteration’, J. Sci. Comput. 54, 684695.
Yu, G., Sapiro, G. and Mallat, S. (2012), ‘Solving inverse problems with piecewise linear estimators: from Gaussian mixture models to structured sparsity’, IEEE Trans. Image Process. 21, 24812499.
Zabih, R. and Woodfill, J. (1994), Non-parametric local transforms for computing visual correspondence. In Proc. 3rd European Conference on Computer Vision: ECCV ’94,Vol. II, Vol. 801 of Lecture Notes in Computer Science, Springer, pp. 151158.
Zach, C., Gallup, D., Frahm, J. M. and Niethammer, M. (2008), Fast global labeling for real-time stereo using multiple plane sweeps. In Vision, Modeling, and Visualization 2008 (Deussen, O., Keim, D. and Saupe, D., eds), IOS Press, pp. 243252.
Zach, C., Pock, T. and Bischof, H. (2007), A duality based approach for realtime TV-L 1 optical flow. In Proc. 29th DAGM Symposium on Pattern Recognition, Vol. 4713 of Lecture Notes in Computer Science, Springer, pp. 214223.
Zavriev, S. K. and Kostyuk, F. V. (1991), The heavy ball method in nonconvex optimization problems. In Software and Models of Systems Analysis (in Russian), Moskov. Gos. Univ., Moscow, pp. 179186, 195., Translation in Comput. Math. Model. 4 (1993), 336–341.
Zeiler, M., Krishnan, D., Taylor, G. and Fergus, R. (2010), Deconvolutional networks. In Proc. IEEE Conference on Computer Vision and Pattern Recognition: CVPR 2010, pp. 25282535.
Zhang, X., Burger, M. and Osher, S. (2011), ‘A unified primal–dual algorithm framework based on Bregman iteration’, J. Sci. Comput. 46, 2046.
Zhang, X., Burger, M., Bresson, X. and Osher, S. (2010), ‘Bregmanized nonlocal regularization for deconvolution and sparse reconstruction’, SIAM J. Imaging Sci. 3, 253276.
Zhu, C., Byrd, R. H., Lu, P. and Nocedal, J. (1997), ‘Algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound-constrained optimization’, ACM Trans. Math. Software 23, 550560.
Zhu, M. and Chan, T. (2008) An efficient primal–dual hybrid gradient algorithm for total variation image restoration. CAM Report 08-34, UCLA.