Electron microscopy is uniquely suited for atomic-resolution imaging of heterogeneous and complex materials, where composition, physical, and electronic structure need to be analyzed simultaneously. Historically, the technique has demonstrated optimal performance at room temperature, since practical aspects such as vibration, drift, and contamination limit exploration at extreme temperature regimes. Conversely, quantum materials that exhibit exotic physical properties directly tied to the quantum mechanical nature of electrons are best studied (and often only exist) at extremely low temperatures. As a result, emergent phenomena, such as superconductivity, are typically studied using scanning probe-based techniques that can provide exquisite structural and electronic characterization, but are necessarily limited to surfaces. In this article, we focus not on the various methods that have been used to examine quantum materials at extremely low temperatures, but on what could be accomplished in the field of quantum materials if the power of electron microscopy to provide structural analysis at the atomic scale was extended to extremely low temperatures.