Static surface shapes of a magnetic fluid volume between two plates in a non-uniform magnetic field are investigated theoretically and experimentally. Abrupt changes and hysteresis of the magnetic fluid surface shape are observed in the experiments when the current in the coil increases and decreases quasi-statically. The necessary and sufficient conditions for a local minimum of the energy functional are derived theoretically. A method to find stable/unstable surface shapes is developed. The ambiguity in the determination of the magnetic fluid surface shape at the same value of the current is shown. It is found that the experimentally observed surface shapes of the given magnetic fluid volume coincide with the shapes obtained numerically, and practically all of them satisfy the derived necessary and sufficient conditions of the minimum energy. The stability curves of the magnetic fluid bridge between the plates are determined experimentally and theoretically.