The origin of residual stresses were studied in both crystalline metallic films and amorphous oxide films made by ion beam assisted deposition (IBAD). Monolithic films of AI2O3 were deposited during bombardment by Ne, Ar or Kr over a narrow range of energies, E, and a wide range of ion-to-atom arrival rate ratios, R and were characterized in terms of composition, thickness, density, crystallinity, microstructure and residual stress. The stress was a strong function of ion beam parameters and gas content and compares to the behavior of other amorphous compounds such as MoSix and WS12.2 With increasing normalized energy (eV/atom), residual stress in crystalline metallic films (Mo, W) increases in the tensile direction before reversing and becoming compressive at high normalized energy. The origin of the stress is most likely due to densification or interstitial generation. Residual stress in amorphous films (Al2O3, MoSix and WSi2.2) is initially tensile and monotonically decreases into the compressive region with increasing normalized energy. The amorphous films also incorporate substantially more gas than crystalline films and in the case of Al2O3 are characterized by a high density of voids. Stress due to gas pressure in existing voids explains neither the functional dependence on gas content nor the magnitude of the observed stress. A more likely explanation for the behavior of stress is gas incorporation into the matrix, where the amount of incorporated gas is controlled by trapping.