The strongest known magnetic fields are found in neutron stars. I briefly discuss how they are inferred from observations, as well as the evidence for their time-evolution. I go on to show how these extremely strong fields are actually weak in terms of their effects on the stellar structure. This is also the case for magnetic stars on the upper main sequence and magnetic white dwarfs, which have similar total magnetic fluxes, perhaps pointing to an evolutionary connection. I suggest that a stable hydromagnetic equilibrium (containing a poloidal and a toroidal field component) could be established soon after the birth of the neutron star, aided by the strong compositional stratification of neutron star matter, and this state is slowly eroded by non-ideal magnetohydrodynamic processes such as beta decays and ambipolar diffusion in the core of the star and Hall drift and breaking of the solid in its crust. Over sufficiently long time scales, the fluid in the neutron star core will behave as if it were barotropic, because, depending on temperature and magnetic field strength, beta decays will keep adjusting the composition to the chemical equilibrium state, or ambipolar diffusion will decouple the charged component from the neutrons. Therefore, the still open question regarding stable hydromagnetic equilibria in barotropic fluids will become relevant for the evolution, at least for magnetar fields, which are likely too strong to be stabilized by the solid crust.