The relationship between atomic structure and elastic properties of grain boundaries is investigated from both discrete and continuum points of view. A heterogeneous continuum model of the boundary is introduced where distinct phases are associated with individual atoms and possess their atomic level elastic moduli determined from the atomistic model. The complete fourth-order tensors of both the atomic-level and the effective elastic moduli are determined, where the latter are defined for sub-blocks from an infinite bicrystal and are calculated here for a relatively small number of atom layers above and below the grain boundary. These effective moduli are determined exactly for the discrete atomistic model, while only estimates from upper and lower bounds can be determined for the continuum model. Comparison between the atomistic results and those for the continuum model establishes the validity of this definition of elastic properties for heterogeneous structures at these scales. Furthermore, these comparisons as well as algebraic properties of the fourth-order tensor of moduli lead to criteria to assess the stability of a given grain boundary structure.