Gigahertz Peaked Spectrum (GPS) radio galaxies are generally thought to be the young counterparts of classical extended radio sources and live in massive ellipticals. GPS sources are vital for studying the early evolution of radio-loud AGN, the trigger of their nuclear activity, and the importance of feedback in galaxy evolution. We study the Parkes half-Jansky sample of GPS radio galaxies of which now all host galaxies have been identified and 80% has their redshifts determined (0.122 < z < 1.539). Analysis of the absolute magnitudes of the GPS host galaxies show that at z > 1 they are on average a magnitude fainter than classical 3C radio galaxies. This suggests that the AGN in young radio galaxies have not yet much influenced the overall properties of the host galaxy. However their restframe UV luminosities indicate that there is a low level of excess as compared to passive evolution models.