The available experimental results have shown that in time-periodic motion the rheology of fluid mud displays complex viscoelastic behaviour. Based on the measured rheology of fluid mud from two field sites, we study the interaction of water waves and fluid mud by a two-layered model in which the water above is assumed to be inviscid and the mud below is viscoelastic. As the fluid-mud layer in shallow seas is usually much thinner than the water layer above, the sharp contrast of scales enables an approximate analytical theory for the interaction between fluid mud and small-amplitude waves with a narrow frequency band. It is shown that at the leading order and within a short distance of a few wavelengths, wave pressure from above forces mud motion below. Over a much longer distance, waves are modified by the accumulative dissipation in mud. At the next order, infragravity waves owing to convective inertia (or radiation stresses) are affected indirectly by mud motion through the slow modulation of the short waves. Quantitative predictions are made for mud samples of several concentrations and from two different field sites.