Boundary layers in space and astrophysical plasmas are the location of complex dynamics where different mechanisms coexist and compete, eventually leading to plasma mixing. In this work, we present fully kinetic particle-in-cell simulations of different boundary layers characterized by the following main ingredients: a velocity shear, a density gradient and a magnetic gradient localized at the same position. In particular, the presence of a density gradient drives the development of the lower-hybrid drift instability (LHDI), which competes with the Kelvin–Helmholtz instability (KHI) in the development of the boundary layer. Depending on the density gradient, the LHDI can even dominate the dynamics of the layer. Because these two instabilities grow on different spatial and temporal scales, when the LHDI develops faster than the KHI an inverse cascade is generated, at least in two dimensions. This inverse cascade, starting at the LHDI kinetic scales, generates structures at scale lengths at which the KHI would typically develop. When that is the case, those structures can suppress the KHI itself because they significantly affect the underlying velocity shear gradient. We conclude that, depending on the density gradient, the velocity jump and the width of the boundary layer, the LHDI in its nonlinear phase can become the primary instability for plasma mixing. These numerical simulations show that the LHDI is likely to be a dominant process at the magnetopause of Mercury. These results are expected to be of direct impact to the interpretation of the forthcoming BepiColombo observations.