Recent direct imaging discoveries of exoplanets have raised new questions about the formation of very low-mass objects in very wide orbits. Several explanations have been proposed, but all of them run into some difficulties, trying to explain all the properties of these objects at once. Here we present the results of a deep adaptive optics imaging survey of 85 stars in the Upper Scorpius young association with Gemini, reaching contrasts of up to 10 magnitudes. In addition to identifying numerous stellar binaries and a few triples, we also found several interesting sub-stellar companions. We discuss the implications of these discoveries, including the possibility of a second pathway to giant planet formation.