The theory of slow viscous flow around a slender body is generalized to the situation where the ambient fluid has a yield stress. The local flow around a cylinder that is moving along or perpendicular to its axis, and rotating, provides a first step in this theory. Unlike for a Newtonian fluid, the nonlinearity associated with the viscoplastic constitutive law precludes one from linearly superposing solutions corresponding to each independent component of motion, and instead demands a full numerical approach to the problem. This is accomplished for the case of a Bingham fluid, along with a consideration of some asymptotic limits in which analytical progress is possible. Since the yield stress of the fluid strongly localizes the flow around the body, the leading-order slender-body approximation is rendered significantly more accurate than the equivalent Newtonian problem. The theory is applied to the sedimentation of inclined cylinders, bent rods and helices, and compared with some experimental data. Finally, the theory is applied to the locomotion of a cylindrical filament driven by helical waves through a viscoplastic fluid.