Vapnik and Chervonenkis proposed in [7] a combinatorial notion of dimension that reflects
the ‘combinatorial complexity’ of families of sets. In the three decades that have passed
since that paper, this notion – the Vapnik–Chervonenkis dimension (VC-dimension) – has
been discovered to be of primal importance in quite a wide variety of topics in both pure
mathematics and theoretical computer science.
In this paper we turn our attention to classes with infinite VC-dimension, a realm
thrown into one big bag by the usual VC-dimension analysis. We identify three levels of
combinatorial complexity of classes with infinite VC-dimension. We show that these levels
fall under the set-theoretic definition of σ-ideals (in particular, each of them is closed
under countable unions), and that they are all distinct. The first of these levels (i.e., the
family of ‘small’ infinite-dimensional classes) coincides with the family of classes which are
non-uniformly PAC-learnable.
Maybe the most surprising contribution of this work is the discovery of an intimate
relation between the VC-dimension of a class of subsets of the natural numbers and the
Lebesgue measure of the set of reals defined when these subsets are viewed as binary
representations of real numbers.
As a by-product, our investigation of the VC-dimension-induced ideals over the reals
yields a new proper extension of the Lebesgue measure. Another offshoot of this work
is a simple result in probability theory, showing that, given any sequence of pairwise
independent events, any random event is eventually independent of the members of the
sequence.