Comprehensive linear stability study of flow in an annular layer of electrolyte driven by the action of the Lorentz force is conducted following the analysis of steady axisymmetric solutions of Suslov et al. (J. Fluid Mech., vol. 828, 2017, pp. 573–600). It is shown that an experimentally observed instability in the form of anticyclonic moving vortices reported in Pérez-Barrera et al. (Magnetohydrodynamics, vol. 51 (2), 2015, pp. 203–213) develops on a background of the basic flow consisting of two tori with the opposite azimuthal vorticity components. It is found that, while the background flow is driven electromagnetically, the appearance of vortices is purely due to hydrodynamic effects: shear of the flow and centrifugal inertial forcing. The current study has also revealed that the unstable two-torus basic flow has a stable single-torus counterpart, both emanating from a saddle-node bifurcation of steady states when the Lorentz force is sufficiently strong. The transition from a one-torus to two-torus flow at weaker forcing is abrupt and leads to the appearance of vortices as soon as it occurs. The ranges of layer depths and Reynolds numbers for which vortices develop on a steady background are determined. Subsequently, weakly nonlinear amplitude expansion is used to find an approximate unsteady solution beyond the saddle-node bifurcation.