Although application of organic fertilizers has become a recommended way for developing sustainable agriculture, it is still unclear whether above-ground and below-ground crops have similar responses to chemical fertilizers (CF) and organic manure (OM) under the same farming conditions. The current study investigated soil quality and crop yield response to fertilization of a double-cropping system with rapeseed (above-ground) and sweet potato (below-ground) in an infertile red soil for 2 years (2014–16). Three fertilizer treatments were compared, including CF, OM and organic manure plus chemical fertilizer (MCF). Organic fertilizers (OM and MCF) increased the yield of both above- and below-ground crops and improved soil biochemical properties significantly. The current study also found that soil-chemical properties were the most important and direct factors in increasing crop yields. Also, crop yield was affected indirectly by soil-biological properties, because no significant effects of soil-biological activities on yield were detected after controlling the positive effects of soil-chemical properties. Since organic fertilizers could not only increase crop yield, but also improve soil nutrients and microbial activities efficiently and continuously, OM application is a reliable agricultural practice for both above- and below-ground crops in the red soils of China.