Multilocation experiments were established to determine the best strategy for using inorganic fertilizer in conservation agriculture (CA) systems that use green manure cover crops, namely sunhemp, velvet bean and cowpea grown in rotation with maize. The objectives of the study were to determine (i) the effect of half and full rates of basal fertilizer on maize and legume biomass yields, (ii) the residual effects of unfertilized, half and fully fertilized green manure legumes on maize grown after the legumes, and (iii) the residual effect of unfertilized, half and fully fertilized green manure legumes combined with basal and topdressing fertilizer on maize yields. Experimental design was a randomized complete block with basal fertilizer as a treatment in the green manure legumes phase. Previously, in the maize phase, green manure legume species were the main treatment with basal fertilizer as a subtreatment (sunhemp, velvet bean and cowpea: 0, 75, 150 kg ha−1 and 0, 50, 100 kg ha−1, respectively). Nitrogen was applied in the maize phase at 0, 23, 46, 69 kg N ha−1 as a sub-subtreatment in Malawi. Results showed that inorganic fertilizer is the most effective when applied to the maize, not green manure legumes. Biomass of green manure legumes, sunnhemp 8084 kg ha−1, velvet bean 7678 kg ha−1 and cowpea 4520 kg ha−1, was not significantly affected by application of basal fertilizer. Maize production increased after the application of green manure legumes with maize-after-maize, maize-after-velvet bean, maize-after-sunnhemp and maize-after-cowpea, yielding 3804, 5440, 5446 and 5339 kg ha−1, respectively. Nitrogen increased maize yield regardless of the previously used green manure legumes species. Our results suggest that farmers should apply fertilizer to maize and grow green manure legumes on residual soil in CA systems. Despite growing green manure legumes, smallholders should apply nitrogen topdressing to maize grown using the green manure legumes in some agro-ecologies.