We compare [$\alpha$/Fe], metallicity, and age distributions of globular clusters in elliptical, lenticular, and spiral galaxies, which we derive from Lick line index measurements. We find a large number of globular clusters in elliptical galaxies that reach significantly higher [$\alpha$/Fe] values ($>0.5$ dex) than clusters in lenticular and spiral galaxies. Most of these highly $\alpha$-enriched globular clusters are old ($t > 8$ Gyr) and exhibit relatively high metallicities up to solar values. Given the lower [$\alpha$/Fe] ratios of the diffuse stellar population in early-type galaxies, our results suggest that the extremely $\alpha$-enhanced globular clusters are members of the the very first generation of star clusters formed, and that their formation epochs likely predate the formation of the majority of stars in giant early-type galaxies.