There is increasing interest in the link between early linguistic skills and later language development. In a longitudinal study, we investigated infants’ (a) ability to use speech sound categories to guide word learning in the habituation-based minimal pair switch task, and (b) early productive vocabulary, related to their concurrent and later language task performance. The participants at Phase 1 were 64 infants aged 16–24 months (25 with familial risk of language/speech impairment), followed up at 27 months (Phase 2) and at 3 years (Phase 3). Phase 1 productive vocabulary was correlated with Phase 2 productive vocabulary, and with concurrent and later (Phase 3) tests of language production and comprehension scores (standardized tool), and phonology. Phase 1 switch task performance was correlated with concurrent productive vocabulary and language production scores, but not by Phase 3. However, a combination of early low vocabulary score and a preference for looking at an already-habituated word–object combination in the switch task may show some promise as an identifier for early speech–language intervention. We discuss how these relations can help us better understand the foundations of word learning.