We present an atomistic model that describes the evolution of ion induced damage ranging from individual defects to continuous amorphous layers. The elementary units used to reproduce the defective zones are Si interstitials, vacancies and the IV pair, which is a local distortion of the Si lattice without any excess or deficit of atoms. More complex defect structures can be formed as these elementary units cluster. The amorphous pockets are treated as agglomerates of IV pairs, whose recrystallization rate depends on the local density of these defects. The local excess or deficit of atoms in the amorphous regions experiences some rearrangement as recrystallization takes place. In sub-amorphizing implants amorphous pockets are disconnected and when they recombine, they leave behind the local excess of Si interstitials and vacancies. When a continuous amorphous layer initially extends to the surface, the excess or deficit atoms within the amorphous layer are swept towards the surface where they are annihilated and only the defects beyond the amorphous-crystalline interface remain.