The IR emission from galaxies is a unique window into multiple aspects of galaxy evolution including star-formation rates, the age of galaxies, and galactic-scale dust processes. However, asymptotic giant branch (AGB) stars continue to introduce uncertainty into stellar population synthesis (SPS) models and limit our ability to interpret the IR light of galaxies. Here we focus on incorporating circumstellar dust around AGB stars in SPS models and understanding the extent to which they influence the IR light of galaxies. We find that the significance of the AGB dust contribution depends on the characteristics of the galaxy. For quiescent galaxies and metal-poor star forming galaxies, circumstellar dust emission can have a large effect, whereas for dusty star-forming galaxies the circumstellar emission is dwarfed by emission from dust in the ISM. The models with circumstellar dust also suggest, in agreement with previous work, that IR colors can be a powerful age diagnostic for older stellar systems. Models such as these will be essential for interpreting data that will be provided by JWST and other next generation IR facilities.