Where did our intelligence come from? That is, what evolutionary drivers caused such specialization in cognition among humans? Only by adopting a comparative approach, considering the brains and cognitive skills of other animal species, can we discover how, when, and even perhaps why human intellectual skills evolved. Here we apply a process of evolutionary reconstruction to ancestors we share with other species, from the earliest primates at 74 Ma (million years ago) to the relatively recent ancestor shared with chimpanzees. Doing so highlights the importance of both social and ecological (nutritional) pressures in evolving intellect. Complex sociality was supported by increased perception, learning, and memory skills, long before the development of any ability to understand other beings as causal agents with independent minds. The latter, we argue, was driven by a need to feed more efficiently in ancestors we share with all living great apes.