The thermal stability and local atomic structures of glassy Fe70M10B20 (M = Hf, Zr, Nb, W and Cr) alloys were analyzed by DSC, ordinary X-ray diffraction and AXS measurements. The random network of the trigonal prism-like structure of (Fe,M)3B with edge-sharing, was identified in all the Fe70M10B20 (M = Hf, Zr, Nb, W and Cr) alloys in spite of the wide variety of thermal stability upon heating. Several unique primary precipitated crystalline phases, such as Fe23B6 type and Fe-M phases, were observed in the alloys exhibiting a high thermal stability. These crystallization reactions require relatively long range rearrangements of the constituents and hence the thermal stability of the glassy phase increases, leading to the appearance of a large supercooled liquid region upon heating. These phenomena may be originated from the difference in the chemical affinity and the atomic size mismatch between M and Fe or B.