We present for the first time the feasibility to recover the stiffness (here shear modulus) distribution of a three-dimensional heterogeneous sample using measured surface displacements and inverse algorithms without making any assumptions about local homogeneities and the stiffness distribution. We simulate experiments to create measured displacements and augment them with noise, significantly higher than anticipated measurement noise. We also test two-dimensional problems in plane strain with multiple stiff inclusions. Our inverse strategy recovers the shear modulus values in the inclusions and background well, and reveals the shape of the inclusion clearly.