In a recent article (Forterre & Pouliquen 2001), we have reported a new instability observed in rapid granular flows down inclined planes that leads to the spontaneous formation of longitudinal vortices. From the experimental observations, we have proposed an instability mechanism based on the coupling between the flow and the granular temperature in rapid granular flows. In order to investigate the relevance of the proposed mechanism, we perform in the present paper a three-dimensional linear stability analysis of steady uniform flows down inclined planes using the kinetic theory of granular flows. We show that in a wide range of parameters, steady uniform flows are unstable under transverse perturbations. The structure of the unstable modes is in qualitative agreement with the experimental observations. This theoretical analysis shows that the kinetic theory is able to capture the formation of longitudinal vortices and validates the instability mechanism.