We report on the selective area heteroepitaxy and facet evolution of AlGaN nanostructures on GaN/sapphire substrate using various mask materials. We also report on the challenges associated with selection of an appropriate mask material for selective area heteroepitaxy of AlGaN with varying Al composition. The shape and the growth rate of the nanostructures are observed to be greatly affected by the mask material. The evolution of the AlGaN nanostructures and Al incorporation were studied exhaustively as a function of growth parameters including temperature, pressure, NH3 flow, total alkyl flow, and TMAl/(TMAl+TMGa) ratio. The growth rate of nanostructures was reduced drastically when higher Al percentage AlGaN nanostructures were grown. The growth rates were increased for higher Al percentage AlGaN using a surfactant, which resulted in a high-quality pyramidal structure. As indicated by high-resolution x-ray diffraction and cathodoluminescence spectroscopy, the composition of Al in the AlGaN nanostructure is significantly different from that of a thin film grown under the same growth conditions.