Research has proved that integrated soil fertility management (ISFM) can increase crop yields at the field and farm scales. However, its uptake by smallholder farmers in Africa is often constrained by lack of technical guidelines on effective starting points and how the different ISFM options can be combined to increase crop productivity on a sustainable basis. A 4-year study was conducted on sandy soils (<10% clay) on smallholder farms in eastern Zimbabwe to assess how sequencing of different ISFM options may lead to incremental gains in soil productivity, enhanced efficiency of resource use, and increase crop yields at field scale. The sequences were primarily based on low-quality organic resources, nitrogen-fixing green manure and grain legumes, and mineral fertilizers. To enable comparison of legume and maize grain yields among treatments, yields were converted to energy (kilocalories) and protein (kg) equivalents. In the first year, ‘Manure-start’, a cattle manure-based sequence, yielded 3.4 t ha−1 of maize grain compared with 2.5 and 0.4 t ha−1 under a woodland litter-based sequence (‘Litter-start’) and continuous unfertilized maize control, respectively. The ‘Manure-start’ produced 12 × 106 kilocalories (kcal); significantly (p < 0.05) out-yielding ‘Litter start’ and a fertilizer-based sequence (‘Fertilizer-start’) by 50%. A soyabean-based sequence, ‘Soya-start’, gave the highest protein production of 720 kg against <450 kg for the other sequencing treatments. In the second year, the sequences yielded an average of 5.7 t ha−1 of maize grain, producing over 19 × 106 kcal and 400 kg of protein. Consequently, the sequences significantly out-performed farmers’ designated poor fields by ~ fivefold. In the third year, ‘Soya-start’ gave the highest maize grain yield of 3.7 t ha−1; translating to 1.5 and 3 times more calories than under farmers’ designated rich and poor fields, respectively. In the fourth year, ‘Fertilizer-start’ produced the highest calories and protein of 14 × 106 kcal and 340 kg, respectively. Cumulatively over 4 years, ‘Manure-start’ and ‘Soya-start’ gave the highest calories and protein, out-performing farmers’ designated rich and poor fields. Sunnhemp (Crotalaria juncea L.)-based sequences, ‘Green-start’ and ‘Fertilizer-start’, recorded the highest gains in plant available soil P of ~ 4 mg kg−1 over the 4-year period. Assessment of P agronomic efficiencies showed significantly more benefits under the ISFM-based sequences than under farmers’ designated rich and poor fields. Based on costs of seed, nutrients and labour, ‘Soya-start’ gave the best net present value over the 4 years, while ‘Fertilizer-start’ was financially the least attractive. Overall, the ISFM-based sequences were more profitable than fields designated as rich and poor by farmers. We concluded that ISFM-based sequences can provide options for farm-level intensification by different categories of smallholder farmers in Southern Africa.