Needles of Juniperus rigida are used in Chinese traditional medicine for the treatment of brucellosis, dropsy, skin disease, and rheumatoid arthritis. This is the first study that reports anatomical structures of the J. rigida needles collected at different altitudes. The most common anatomical, phytochemical, and histochemical techniques and methods are used. The results show that anatomical structures and chemical composition change significantly at different altitudes. The main anatomical characters are significant xeromorphic structures (thick epidermis, hypodermis, and cuticle), a stomatal band, a developed vascular bundle, and a marginal resin duct. The xeromorphic structures become more pronounced with increasing altitude. The phytochemical and histochemical results demonstrate that the content of the main chemical compounds (phenols and terpenoids) basically increases at a higher elevation. Histochemical analysis localizes the phenols in epidermal cells, sponge tissue, endothelial layer cells, and stomatal bands, and the terpenoids in palisade tissue, sponge tissue, and the edge of the resin duct. This work reveals the relation between anatomy and chemistry in J. rigida needles, contributes to the quality control of its ethno-medicine, and provides the evidence to develop the commercial cultivation.