We have presented a mathematical framework that results in the specification of the broadest possible class of linear stochastic processes. The remarkable aspect is that these continuous-domain processes are either Gaussian or sparse, as a direct consequence of the theory. While the formulation relies on advanced mathematical concepts (distribution theory, functional analysis), the underlying principles are simple and very much in line with the traditional methods of statistical signal processing. The main point is that one can achieve a whole variety of sparsity patterns by combining relatively simple building blocks: non-Gaussian white-noise excitations and suitable integral operators. This results in non-Gaussian processes whose properties are compatible with the modern sparsity-based paradigm for signal processing. Yet, the proposed class of models is also backward-compatible with the linear theory of signal processing (LMMSE = linear minimum mean square estimation) since the correlation structure of the processes remains the same as in the traditional Gaussian case – the processes are ruled by the same stochastic differential equations and it is only the driving terms (innovations) that differ in their level of sparsity. On the theoretical front, we have highlighted the crucial role of the generalized B-spline function βL – in one-to-one relation with the whitening operator L – that provides the functional link between the continuous-domain specification of the stochastic model and the discrete-domain handling of the sample values. We have also shown that these processes admit a sparse wavelet decomposition whenever the wavelet is matched to the whitening operator.