Experimental studies of laminar plumes descending under gravity into stably stratified environments have shown the existence of a critical injection velocity beyond which the plume exhibits a bifurcation to a coiling instability in three dimensions or a sinuous instability in a Hele-Shaw flow. In addition, flow visualization has shown that, prior to the onset of the instability, a stable base flow is established in which the plume penetrates to a depth significantly smaller than the neutral buoyancy depth. Moreover, the fresh water that is viscously entrained by the plume recirculates within a ‘conduit’ whose boundary with the background stratification appears sharp. Beyond the bifurcation, the buckling plume takes the form of a travelling wave of varying amplitude, confined within the conduit, which disappears at the penetration depth. To determine the mechanisms underlying these complex phenomena, which take place at a strikingly low Reynolds number but a high Schmidt number, we study here a two-dimensional arrangement, as it is perhaps the simplest system which possesses all the key experimental features. Through a combination of numerical and analytical approaches, a scaling law is found for the plume's penetration depth within the base flow (i.e. the flow where the instability is either absent or artificially suppressed), and the horizontal cross-stream velocity and concentration profile outside the plume are determined from an asymptotic analysis of a simplified model. Direct numerical simulations show that, with increasing flow rate, a sinuous global mode is destabilized giving rise to the self-sustained oscillations as in the experiment. The sinuous instability is shown to be a consequence of the baroclinic generation of vorticity, due to the strong horizontal gradients at the edge of the conduit, a mechanism that is relevant even at very low Reynolds numbers. Despite the strength of this instability, the penetration depth is not significantly affected by it, instead being determined by the properties of the plume in the vicinity of the source. This scenario is confirmed by a local stability analysis. A finite region of local absolute instability is found near the source for sinuous modes prior to the onset of the global instability. Sufficiently far from the source the flow is locally stable. Near the onset of the global instability, varicose modes are also found to be locally, but only convectively, unstable.