Influences of annealing on the structure of mesoporous silica loaded with silver (Ag) nanoparticles, and on the coarsening of Ag particles within pores of the host were investigated from isothermal sorption. Doping a small amount of Ag nanoparticles into pores of silica and subsequent annealing decreases the measured values of specific surface area and pore volume of porous silica significantly. This is attributed to the presence and coarsening of Ag particles within pores or channels between pores, which result in more and more isolated and unmeasured free spaces. The measured value of a specific surface area for the doped samples cannot represent the real value, which is, in fact, unable to be measured directly. During additional annealing, Ag particles within silica coarsen mainly according to the mechanism of formation of Ag adatoms on pore wall and diffusion of the adatoms along with pore walls. Only the larger particles located in the larger pores can continuously grow. The smaller particles and those located in the channels or pores with smaller dimension will disappear. The activation energy of the ripening process was estimated to be about 0.60 eV, and the migration barrier of Ag adatom on the pore wall of silica is about 0.10 eV.