Understanding the two-way interactions between finite-size solid particles and a wall-bounded turbulent flow is crucial in a variety of natural and engineering applications. Previous experimental measurements and particle-resolved direct numerical simulations revealed some interesting phenomena related to particle distribution and turbulence modulation, but their in-depth analyses are largely missing. In this study, turbulent channel flows laden with neutrally buoyant finite-size spherical particles are simulated using the lattice Boltzmann method. Two particle sizes are considered, with diameters equal to 14.45 and 28.9 wall units. To understand the roles played by the particle rotation, two additional simulations with the same particle sizes but no particle rotation are also presented for comparison. Particles of both sizes are found to form clusters. Under the Stokes lubrication corrections, small particles are found to have a stronger preference to form clusters, and their clusters orientate more in the streamwise direction. As a result, small particles reduce the mean flow velocity less than large particles. Particles are also found to result in a more homogeneous distribution of turbulent kinetic energy (TKE) in the wall-normal direction, as well as a more isotropic distribution of TKE among different spatial directions. To understand these turbulence modulation phenomena, we analyse in detail the total and component-wise volume-averaged budget equations of TKE with the simulation data. This budget analysis reveals several mechanisms through which the particles modulate local and global TKE in the particle-laden turbulent channel flow.