In this communication, we offer a theoretical explanation for the results of recent experiments that examine the stress response of a dilute suspension of bacteria (wild-type E. coli) subjected to step changes in the shear rate (Lopez et al., Phys. Rev. Lett., vol. 115, 2015, 028301). The observations include a regime of negative apparent shear viscosities. We start from a kinetic equation that describes the evolution of the single-bacterium orientation probability density under the competing effects of an induced anisotropy by the imposed shear, and a return to isotropy on account of stochastic relaxation mechanisms (run-and-tumble dynamics and rotary diffusion). We then obtain analytical predictions for the stress response, at leading order, of a dilute bacterial suspension subject to a weak but arbitrary time-dependent shear rate profile. While the predicted responses for a step-shear compare well with the experiments for typical choices of the microscopic parameters that characterize the swimming motion of a single bacterium, use of actual experimental values leads to significant discrepancies. The incorporation of a distribution of run times leads to a better agreement with observations.