The relative dispersion of pairs of inertial point particles in incompressible, homogeneous and isotropic three-dimensional turbulence is studied by means of direct numerical simulations at two values of the Taylor-scale Reynolds number Reλ ~ 200 and Reλ ~ 400, corresponding to resolutions of 5123 and 20483 grid points, respectively. The evolution of both heavy and light particle pairs is analysed by varying the particle Stokes number and the fluid-to-particle density ratio. For particles much heavier than the fluid, the range of available Stokes numbers is St ∈ [0.1 : 70], while for light particles the Stokes numbers span the range St ∈ [0.1 : 3] and the density ratio is varied up to the limit of vanishing particle density. For heavy particles, it is found that turbulent dispersion is schematically governed by two temporal regimes. The first is dominated by the presence, at large Stokes numbers, of small-scale caustics in the particle velocity statistics, and it lasts until heavy particle velocities have relaxed towards the underlying flow velocities. At such large scales, a second regime starts where heavy particles separate as tracers' particles would do. As a consequence, at increasing inertia, a larger transient stage is observed, and the Richardson diffusion of simple tracers is recovered only at large times and large scales. These features also arise from a statistical closure of the equation of motion for heavy particle separation that is proposed and is supported by the numerical results. In the case of light particles with high density ratio, strong small-scale clustering leads to a considerable fraction of pairs that do not separate at all, although the mean separation increases with time. This effect strongly alters the shape of the probability density function of light particle separations.