The microstructure of In(As,Bi)/GaAs heterostructures grown by low-temperature molecular beam epitaxy with special attention to the interfaces was studied by scanning/transmission electron microscopy, energy dispersive X-ray microanalysis, and X-ray diffraction and reflectivity. Two samples grown at similar conditions with and without the presence of the Bi-contained layer, formed at 350 °C, are considered. These samples were jointly analyzed to clarify Bi influence on the crystal structure. Two types of QD-like clusters at the GaAs/In(As,Bi) interface were found. The first type exhibited a zinc blend crystal structure, which is typical for A3B5 semiconductors. The second type adopted a tetragonal PbO crystal structure and was found in different orientations. The joint analysis by electron microscopy and X-ray methods demonstrated that the incorporation of Bi atoms into the InAs layer leads to the strain relaxation at the interface in the growth direction. According to electron microscopy data, this strain release is more pronounced around the clusters of the second type.