Atomically resolved imaging of materials enabled by the advent of aberration-corrected scanning transmission electron microscopy (STEM) has become a mainstay of modern materials science. However, much of the wealth of quantitative information contained in the fine details of atomic structure or spectra remains largely unexplored. In this article, we discuss new opportunities enabled by physics-informed big data and machine learning technologies to extract physical information from static and dynamic STEM images, ranging from statistical thermodynamics of alloys to kinetics of solid-state reactions at a single defect level. The synergy of deep-learning image analytics and real-time feedback further allows harnessing beam-induced atomic and bond dynamics to enable direct atom-by-atom fabrication. Examples of direct atomic motion over mesoscopic distances, engineered doping at selected lattice sites, and assembly of multiatomic structures are reviewed. These advances position the scanning transmission electron microscope to transition from a mere imaging tool toward a complete nanoscale laboratory for exploring electronic, phonon, and quantum phenomena in atomically engineered structures.