Concentrated solid-solution alloys (CSAs) demonstrate excellent mechanical properties and promising irradiation resistance depending on their compositions. Existing experimental and simulation results indicate that their heterogeneous structures induced by the random arrangement of different elements are one of the most important reasons responsible for their outstanding properties. Nevertheless, the details of this heterogeneity remain unclear. Specifically, which properties induced by heterogeneity are most relevant to their irradiation response? In this work, we scrutinize the role of heterogeneity in CSAs played in damage evolution in different aspects through atomistic simulations, including lattice misfit, thermodynamic mixing, point defect energetics, point defect diffusion, and dislocation properties. Our results reveal that structural parameters, such as lattice misfit and enthalpy of mixing, are generally not suitable to assess their irradiation response under cascade conditions. Instead, atomic-level defect properties are the keys to understand defect evolution in CSAs. Therefore, tuning chemical disorder to tailor defect properties is a possible way to further improve the irradiation performance of CSAs.