Loss functions with a large number of saddle points are one of the major obstacles for training modern machine learning (ML) models efficiently. First-order methods such as gradient descent (GD) are usually the methods of choice for training ML models. However, these methods converge to saddle points for certain choices of initial guesses. In this paper, we propose a modification of the recently proposed Laplacian smoothing gradient descent (LSGD) [Osher et al., arXiv:1806.06317], called modified LSGD (mLSGD), and demonstrate its potential to avoid saddle points without sacrificing the convergence rate. Our analysis is based on the attraction region, formed by all starting points for which the considered numerical scheme converges to a saddle point. We investigate the attraction region’s dimension both analytically and numerically. For a canonical class of quadratic functions, we show that the dimension of the attraction region for mLSGD is $\lfloor (n-1)/2\rfloor$ , and hence it is significantly smaller than that of GD whose dimension is $n-1$ .

As a continuation of previous work of the first author with Ranjbar [‘A variational inequality in complete CAT(0) spaces’, J. Fixed Point Theory Appl. 17 (2015), 557–574] on a special form of variational inequalities in Hadamard spaces, in this paper we study equilibrium problems in Hadamard spaces, which extend variational inequalities and many other problems in nonlinear analysis. In this paper, first we study the existence of solutions of equilibrium problems associated with pseudo-monotone bifunctions with suitable conditions on the bifunctions in Hadamard spaces. Then, to approximate an equilibrium point, we consider the proximal point algorithm for pseudo-monotone bifunctions. We prove existence of the sequence generated by the algorithm in several cases in Hadamard spaces. Next, we introduce the resolvent of a bifunction in Hadamard spaces. We prove convergence of the resolvent to an equilibrium point. We also prove $\triangle$-convergence of the sequence generated by the proximal point algorithm to an equilibrium point of the pseudo-monotone bifunction and also the strong convergence under additional assumptions on the bifunction. Finally, we study a regularization of Halpern type and prove the strong convergence of the generated sequence to an equilibrium point without any additional assumption on the pseudo-monotone bifunction. Some examples in fixed point theory and convex minimization are also presented.

This paper presents topology optimization of capillary, the typical two-phase flow with immiscible fluids, where the level set method and diffuse-interface model are combined to implement the proposed method. The two-phase flow is described by the diffuse-interface model with essential no slip condition imposed on the wall, where the singularity at the contact line is regularized by the molecular diffusion at the interface between two immiscible fluids. The level set method is utilized to express the fluid and solid phases in the flows and the wall energy at the implicit fluid-solid interface. Based on the variational procedure for the total free energy of two-phase flow, the Cahn-Hilliard equations for the diffuse-interface model are modified for the two-phase flow with implicit boundary expressed by the level set method. Then the topology optimization problem for the two-phase flow is constructed for the cost functional with general formulation. The sensitivity analysis is implemented by using the continuous adjoint method. The level set function is evolved by solving the Hamilton-Jacobian equation, and numerical test is carried out for capillary to demonstrate the robustness of the proposed topology optimization method. It is straightforward to extend this proposed method into the other two-phase flows with two immiscible fluids.

The pricing model for American lookback options can be characterised as a two-dimensional free boundary problem. The main challenge in this problem is the free boundary, which is also the main concern for financial investors. We use a standard technique to reduce the pricing model to a one-dimensional linear complementarity problem on a bounded domain and obtain a corresponding variational inequality. The inequality is discretised by finite differences and finite elements in the temporal and spatial directions, respectively. By enforcing inequality constraints related to the options using Lagrange multipliers, the discretised variational inequality is reformulated as a set of semi-smooth equations, which are solved by a primal-dual active set method. One of the major advantages of our algorithm is that we can obtain the option values and the free boundary simultaneously, and numerical simulations show that our approach is as efficient as some other methods.

The hybrid variational model for restoration of texture images corrupted by blur and Gaussian noise we consider combines total variation regularisation and a fractional-order regularisation, and is solved by an alternating minimisation direction algorithm. Numerical experiments demonstrate the advantage of this model over the adaptive fractional-order variational model in image quality and computational time.

We consider the total curvature of graphs of curves in high-codimension Euclidean space. We introduce the corresponding relaxed energy functional and prove an explicit representation formula. In the case of continuous Cartesian curves, i.e. of graphs cu of continuous functions u on an interval, we show that the relaxed energy is finite if and only if the curve cu has bounded variation and finite total curvature. In this case, moreover, the total curvature does not depend on the Cantor part of the derivative of u. We treat the wider class of graphs of one-dimensional functions of bounded variation, and we prove that the relaxed energy is given by the sum of the length and total curvature of the new curve obtained by closing the holes in cu generated by jumps of u with vertical segments.

We propose a new two-phase method for reconstruction of blurred images corrupted by impulse noise. In the first phase, we use a noise detector to identify the pixels that are contaminated by noise, and then, in the second phase, we reconstruct the noisy pixels by solving an equality constrained total variation minimization problem that preserves the exact values of the noise-free pixels. For images that are only corrupted by impulse noise (i.e., not blurred) we apply the semismooth Newton's method to a reduced problem, and if the images are also blurred, we solve the equality constrained reconstruction problem using a first-order primal-dual algorithm. The proposed model improves the computational efficiency (in the denoising case) and has the advantage of being regularization parameter-free. Our numerical results suggest that the method is competitive in terms of its restoration capabilities with respect to the other two-phase methods.

Retinex theory explains how the human visual system perceives colors. The goal of retinex is to decompose the reflectance and the illumination from the given images and thereby compensating for non-uniform lighting. The existing methods for retinex usually use a single image with a fixed exposure to restore the reflectance of the image. In this paper, we propose a variational model for retinex problem by utilizing multi-exposure images of a given scene. The existence and uniqueness of the solutions of the proposed model have been elaborated. An alternating minimization method is constructed to solve the proposed model and its convergence is also demonstrated. The experimental results show that the proposed method is effective for reflectance recovery in retinex problem.

This paper introduces a two-stage model for multi-channel image segmentation, which is motivated by minimal surface theory. Indeed, in the first stage, we acquire a smooth solution u from a convex variational model related to minimal surface property and different data fidelity terms are considered. This minimization problem is solved efficiently by the classical primal-dual approach. In the second stage, we adopt thresholding to segment the smoothed image u. Here, instead of using K-means to determine the thresholds, we propose a more stable hill-climbing procedure to locate the peaks on the 3D histogram of u as thresholds, in the meantime, this algorithm can also detect the number of segments. Finally, numerical results demonstrate that the proposed method is very robust against noise and superior to other image segmentation approaches.

In this work, we develop an adaptive algorithm for solving elliptic optimal control problems with simultaneously appearing state and control constraints. The algorithm combines a Moreau-Yosida technique for handling state constraints with a semi-smooth Newton method for solving the optimality systems of the regularized sub-problems. The state and adjoint variables are discretized using continuous piecewise linear finite elements while a variational discretization concept is applied for the control. To perform the adaptive mesh refinements cycle we derive local error estimators which extend the goal-oriented error approach to our setting. The performance of the overall adaptive solver is assessed by numerical examples.

In this work, we develop a minimum action method (MAM) with optimal linear time scaling, called tMAM for short. The main idea is to relax the integration time as a functional of the transition path through optimal linear time scaling such that a direct optimization of the integration time is not required. The Feidlin-Wentzell action functional is discretized by finite elements, based on which h-type adaptivity is introduced to tMAM. The adaptive tMAM does not require reparametrization for the transition path. It can be applied to deal with quasi-potential: 1) When the minimal action path is subject to an infinite integration time due to critical points, tMAM with a uniform mesh converges algebraically at a lower rate than the optimal one. However, the adaptive tMAM can recover the optimal convergence rate. 2) When the minimal action path is subject to a finite integration time, tMAM with a uniform mesh converges at the optimal rate since the problem is not singular, and the optimal integration time can be obtained directly from the minimal action path. Numerical experiments have been implemented for both SODE and SPDE examples.

An inverse problem of identifying inhomogeneity or crack in the workpiece made of nonlinear magnetic material is investigated. To recover the shape from the local measurements, a piecewise constant level set algorithm is proposed. By means of the Lagrangian multiplier method, we derive the first variation w.r.t the piecewise constant level set function and obtain the descent direction by the adjoint variable method. Numerical results show the robustness and effectiveness of our algorithm applied to reconstruct some complex shapes.

An efficient numerical method is proposed for the valuation of American options via the Black-Scholes variational inequality. A far field boundary condition is employed to truncate the unbounded domain problem to produce the bounded domain problem with the associated variational inequality, to which our finite element method is applied. We prove that the matrix involved in the finite element method is symmetric and positive definite, and solve the discretized variational inequality by the projection and contraction method. Numerical experiments are conducted that demonstrate the superior performance of our method, in comparison with earlier methods.

In this paper, we propose a fast proximity point algorithm and apply it to total variation (TV) based image restoration. The novel method is derived from the idea of establishing a general proximity point operator framework based on which new first-order schemes for total variation (TV) based image restoration have been proposed. Many current algorithms for TV-based image restoration, such as Chambolle’s projection algorithm, the split Bregman algorithm, the Bermúdez-Moreno algorithm, the Jia-Zhao denoising algorithm, and the fixed point algorithm, can be viewed as special cases of the new first-order schemes. Moreover, the convergence of the new algorithm has been analyzed at length. Finally, we make comparisons with the split Bregman algorithm which is one of the best algorithms for solving TV-based image restoration at present. Numerical experiments illustrate the efficiency of the proposed algorithms.

In this paper we present recent developments concerning a Cell-Centered Arbitrary Lagrangian Eulerian (CCALE) strategy using the Moment Of Fluid (MOF) interface reconstruction for the numerical simulation of multi-material compressible fluid flows on unstructured grids in cylindrical geometries. Especially, our attention is focused here on the following points. First, we propose a new formulation of the scheme used during the Lagrangian phase in the particular case of axisymmetric geometries. Then, the MOF method is considered for multi-interface reconstruction in cylindrical geometry. Subsequently, a method devoted to the rezoning of polar meshes is detailed. Finally, a generalization of the hybrid remapping to cylindrical geometries is presented. These explorations are validated by mean of several test cases using unstructured grid that clearly illustrate the robustness and accuracy of the new method.

Total variation (TV) and wavelet L1 norms have often been used as regularization terms in image restoration and reconstruction problems. However, TV regularization can introduce staircase effects and wavelet regularization some ringing artifacts, but hybrid TV and wavelet regularization can reduce or remove these drawbacks in the reconstructed images. We need to compute the proximal operator of hybrid regularizations, which is generally a sub-problem in the optimization procedure. Both TV and wavelet L1 regularisers are nonlinear and non-smooth, causing numerical difficulty. We propose a dual iterative approach to solve the minimization problem for hybrid regularizations and also prove the convergence of our proposed method, which we then apply to the problem of MR image reconstruction from highly random under-sampled k-space data. Numerical results show the efficiency and effectiveness of this proposed method.

We propose a new nonrigid registration algorithm which is based on the optimal control approach. In our previously proposed methods, the Jacobian determinant and the curl vector were used as control functions. In this algorithm, we use a new set of control functions. A main advantage of using the new controls is that the positivity and normalization of the Jacobian determinant are satisfied automatically. Numerical results on large deformation brain images are provided to show the accuracy and efficiency of the algorithm.

This paper proposes a variational binary level set method for shape and topology optimization of structural. First, a topology optimization problem is pre-sented based on the level set method and an algorithm based on binary level set method is proposed to solve such problem. Considering the difficulties of coordination between the various parameters and efficient implementation of the proposed method, we present a fast algorithm by reducing several parameters to only one parameter, which would substantially reduce the complexity of computation and make it easily and quickly to get the optimal solution. The algorithm we constructed does not need to re-initialize and can produce many new holes automatically. Furthermore, the fast algorithm allows us to avoid the update of Lagrange multiplier and easily deal with constraints, such as piecewise constant, volume and length of the interfaces. Finally, we show several optimum design examples to confirm the validity and efficiency of our method.