While there have been growing calls for historians to listen to the past, there are also significant barriers to integrating music in particular into broader historical practice. This article reflects on both the gains and difficulties of this integration, moving from an interrogation of the category of music to three case studies. These concern musical terms, compositional practices and cultures from the fifteenth to eighteenth centuries, revisiting some key debates in musicology: first, the highly charged language of sweetness deployed in the fifteenth century; second, connections discerned in nineteenth-century music history between medieval polyphony and contemporary attitudes towards time and authority; and, third, debate over the anti-Jewish implications of Handel's music, which we approach through his Dixit Dominus and a history of psalm interpretation stretching back to late antiquity. Through these case studies, we suggest the contribution of music to necessarily interdisciplinary fields including the study of temporality and emotions, but also explore how a historical hermeneutic with a long pedigree – ‘diversity of times’ (diversitas temporum) – might help to reframe arguments about musical interpretation. The article concludes by arguing that the very difficulty and slipperiness of music as a source can encourage properly reflective historical practice.