Let
$S=\{p_1, \ldots , p_r,\infty \}$ for prime integers
$p_1, \ldots , p_r.$ Let X be an S-adic compact nilmanifold, equipped with the unique translation-invariant probability measure
$\mu .$ We characterize the countable groups
$\Gamma $ of automorphisms of X for which the Koopman representation
$\kappa $ on
$L^2(X,\mu )$ has a spectral gap. More specifically, let Y be the maximal quotient solenoid of X (thus, Y is a finite-dimensional, connected, compact abelian group). We show that
$\kappa $ does not have a spectral gap if and only if there exists a
$\Gamma $-invariant proper subsolenoid of Y on which
$\Gamma $ acts as a virtually abelian group,