A yield stress is added to Taylor’s (Proc. R. Soc. Lond. A, vol. 209, 1951, pp. 447–461) model of a two-dimensional flexible sheet swimming through a viscous fluid. Both transverse waves along the sheet, as in Taylor’s original model, and longitudinal waves are considered as means of locomotion. In each case, numerical solutions are provided over a range of the two key parameters of the problem: the wave amplitude relative to the wavelength and a Bingham number which describes the strength of the yield stress. The numerical solutions are supplemented with discussions of various limits of the problem in which analytical progress is possible. When the yield stress is large, the swimming speed for low wave amplitude is exactly double that for a Newtonian fluid, for either type of wave.