Convex duality is a powerful framework for solving non-smooth optimal control problems. However, for problems set in non-reflexive Banach spaces such as L1(Ω) or BV(Ω), the dual problem is formulated in a space which has difficult measure theoretic structure. The predual problem, on the other hand, can be formulated in a Hilbert space and entails the minimization of a smooth functional with box constraints, for which efficient numerical methods exist. In this work, elliptic control problems with measures and functions of bounded variation as controls are considered. Existence and uniqueness of the corresponding predual problems are discussed, as is the solution of the optimality systems by a semismooth Newton method. Numerical examples illustrate the structural differences in the optimal controls in these Banach spaces, compared to those obtained in corresponding Hilbert space settings.