In this paper, we propose a fast proximity point algorithm and apply it to total variation (TV) based image restoration. The novel method is derived from the idea of establishing a general proximity point operator framework based on which new first-order schemes for total variation (TV) based image restoration have been proposed. Many current algorithms for TV-based image restoration, such as Chambolle’s projection algorithm, the split Bregman algorithm, the Bermúdez-Moreno algorithm, the Jia-Zhao denoising algorithm, and the fixed point algorithm, can be viewed as special cases of the new first-order schemes. Moreover, the convergence of the new algorithm has been analyzed at length. Finally, we make comparisons with the split Bregman algorithm which is one of the best algorithms for solving TV-based image restoration at present. Numerical experiments illustrate the efficiency of the proposed algorithms.