Lithotripsy methods show relatively low efficiency in the fragmentation of sialoliths compared with the success rates achieved in the destruction of renal calculi. However, the information available on the mechanical behavior of sialoliths is limited and their apparently tougher response is not fully understood. This work evaluates the hardness and Young’s modulus of sialoliths at different scales and analyzes specific damage patterns induced in these calcified structures by ultrasonic vibrations, pneumoballistic impacts, shock waves, and laser ablation. A clear correlation between local mechanical properties and ultrastructure/chemistry has been established: sialoliths are composite materials consisting of hard and soft components of mineralized and organic nature, respectively. Ultrasonic and pneumoballistic reverberations damage preferentially highly mineralized regions, leaving relatively unaffected the surrounding organic matter. In contrast, shock waves leach the organic component and lead to erosion of the overall structure. Laser ablation destroys homogeneously the irradiated zones regardless of the mineralized/organic nature of the underlying ultrastructure; however, damage is less extensive than with mechanical methods. Overall, the present results show that composition and internal structure are key features behind sialoliths’ comminution behavior and that the organic matter contributes to reduce the therapeutic efficiency of lithotripsy methods.