The methods of formal matched asymptotics are used to investigate the motion of a vortex in shallow inviscid fluid of varying depth and zero Froude number in the limit as the vortex core radius tends to zero. To leading order the vortex is driven by local gradients in the logarithm of the depth along an isobath (or depth contour). A further term in the vortex velocity is calculated in which effects arising from the global bottom topography, other vortices and the vortex core structure appear. The evolution of the vortex core structure is then calculated. A point-vortex model is formulated which describes the motion of a number of small vortices in terms of the bottom topography, the inviscid flows around the vortices and their evolving core structure. A numerical method for solving this model is discussed and finally some numerical simulations of the motion of vortex pairs over a varying bottom topography are presented.