A vertically standing freely-rotating ellipsoidal vortex of uniform anomalous potential vorticity in a rotating stratified fluid under quasi-geostrophic conditions of small Rossby and Froude numbers steadily rotates without change of form. The vortex can have arbitrary axis lengths, but must have one axis parallel to the vertical $z$-axis along the direction of gravity. The rotation rate is proportional to the potential vorticity anomaly but otherwise depends on only two independent aspect ratios characterizing the shape of the vortex. The linear stability of this class of vortex equilibria was first determined semi-analytically more than a decade ago. It was found that vortices are unstable over a wide range of the parameter space and are stable only when strongly oblate and of nearly circular cross-section.

New results, presented here, using a complementary approach and backed by nonlinear simulations of the full quasi-geostrophic equations indicate that these ellipsoidal vortices are in fact stable over a much wider range of parameter space. In particular, a mode previously thought to be unstable over much of the parameter space is evidently stable. Moreover, we have determined that this mode is just the difference between two neighbouring equilibrium states having slightly different horizontal aspect ratios; hence, this mode must be neutrally stable. Agreement is found for all other modes. However, by an independent analysis considering only ellipsoidal (though time-varying) disturbances, we have identified one unstable mode as purely ellipsoidal, i.e. it does not change the form of the ellipsoid, only its shape. Under this instability, the vortex quasi-periodically tilts over while undergoing mild changes in shape.

The range of parameters leading to non-ellipsoidal instabilities turns out to be narrow, with instability principally occurring for highly eccentric (horizontally squashed, prolate) vortices. The long-term fate of these instabilities is examined by nonlinear contour-dynamical simulations. These reveal a wealth of complex phenomena such as the production of a sea of small-scale vortices, yet, remarkably, the dominant vortex often tends to relax to a stable rotating ellipsoid.