Recently cosmology with unstable particles has attracted much attention as a possible solution to several important problems in the present universe. In this scenario, however, nonlinear structures in the universe would not easily form since their binding energy would decrease gradually in the course of the decay of unstable particles. Thus this scenario is stringently constrained by the galaxy formation problem. In order to obtain the constraints quantitatively, we carried out numerical calculations of the evolution of density perturbations after the time of recombination. Universe is assumed to consist of three components; unstable X-particles, its non-radiative decay product (massless particles) and baryons. Initial conditions are specified by the density contrast of X-particles δX,i and the baryon density parameter Ωb,i. Resultant baryon density contrast at Z = 4 (epoch of galaxy formation) and Z = 0 (present) is shown below for Ωb,i = 0.005.