This research sought to link Andean soil knowledge and farmer categorization of soil fertility to soil science characterization of soils, and use these to understand the impacts of phosphorus (P) fertilization of legumes using rock phosphate and soluble P fertilizer in 17 smallholder-managed sites with varying soil properties. We found that farmer high/low categorization of soils corresponded to soil P fertility and distance from farmer dwellings. Measures of soil P fertility also were inversely related to mycorrhizal colonization of vetch roots and directly related to the potential for P release by legume residues (C:P ratio). However, particular soil properties (texture and calcium phosphate pools) were better in explaining the response of legume biological nitrogen fixation to P addition, with maximal impacts in low-clay soils and soils with low calcium phosphates, as assessed with a dilute HCl extraction. In these conducive contexts, legume BNF increased 67 and 150% for RP and TSP, respectively (p < 0.05), with similar increases in biomass P stocks that showed promise for soil regeneration. In low-P fields, added P increased potential P release from legume residues via lower C:P ratios (p < 0.05). Percent cover of soil increased from 35% in the control to 45 and 55% with RP and TSP (p < 0.05), which shows potential for reduced erosion risk under P fertilization of legumes. We discuss results with respect to the integration of local farmer knowledge systems with exogenous scientific knowledge.