Gynodioecy, a genetic dimorphism of females and hermaphrodites, is pertinent to an understanding of the evolution of plant gender, mating and genetic variability. Classical models of nuclear gynodioecy attribute the maintenance of the dimorphism to frequency-dependent selection in which the female phenotype has a fitness advantage at low frequency owing to a doubled ovule fertility. Here, I analyse explicit genetic models of nuclear gynodioecy that expand on previous work by allowing partial male sterility in combination with either fixed or dynamically evolving mutational inbreeding depression. These models demonstrate that partial male sterility causes fitness underdominance at the mating locus, which can prevent the spread of females. However, if partial male sterility is compensated by a change in selfing rate, overdominance at the mating locus can cause the spread of females. Overdominance at introduction of the male sterility allele can be caused by high inbreeding depression and a lower selfing rate in the heterozygote, by purging of mutations by a higher selfing rate in the heterozygote, and by low inbreeding depression and a higher selfing rate in the heterozygote. These processes might be of general importance in the maintenance of mating polymorphisms in plants.