Using cathodoluminescence measurements, we studied the excitation of rare earth ions in LiNbO3 and compare it with previously studied Eu ions in GaN. We find that in both hosts and all dopants the most efficient excitation pathways involves a defect traps that capture the energy from the electron-hole pairs created in the host by E-beam irradiation. Even then the excitation efficiency in LiNbO3 is very low. Moreover, only a small fraction (<10-4) of the RE ions that are optically active can be excited through this pathway. We explain this behavior by a negligible direct excitation and a strong dependence of the excitation efficiency on the distance between the defect trap and the rare earth ion. The density of the defect traps is small such that many rare earth ions are far away from traps.