This study investigates how consistent genetic factors are, as measured by heritability estimates (h2), in the leisure-time physical activity index (LTPAI) and sport participation index (SPI) from early (10–14 yrs) to late adolescence (15–19 yrs). The sample comprises 12,385 subjects from 3,378 Portuguese nuclear families. Height and weight were measured and body mass index (BMI) was calculated, and the LTPAI and SPI were estimated by questionnaire. Socioeconomic status (SES) was assessed by parental occupation. Analyses were done using S.A.G.E. software. Our results showed that h2 estimates for the LTPAI and SPI in the two age groups (10–14 yrs and 15–19 yrs) were stable: for the LTPAI, h2 = 0.297 and 0.322, respectively; and for the SPI, h2 = 0.413 and 0.428, respectively. Sibling correlations and environmental correlations are higher in the younger age group for both the LTPAI and the SPI. Spousal correlations are higher in the younger age group for the LTPAI and lower for the SPI than the older group. Parent–offspring correlations are similar in both age groups for the LTPAI and SPI. In conclusion, the influence of genetic factors on physical activity and sport participation remains stable across age in adolescence. However, variation in sibling correlations — in particular, environmental correlations — was observed. These findings suggest that shared/non-shared environmental factors express different degrees of importance across age. Future intervention programs aiming to promote change in behaviors need to consider these results to bring about positive changes in physical activity and sport participation behaviors within the family setting.