Oxide inclusions such as gray spots are the main defects caused by rail flash butt welding (FBW). An appropriate temperature field and upsetting process are essential for the extrusion of joint impurities. This study constructed a thermomechanical coupling model for the solid-state upsetting process of rail FBW through a combination of finite element simulation and experiment. Subsequently, the effects of different temperature fields and upsetting parameters on the extrusion behavior of impurities were studied. The results show that when the lateral deformation of the joint is not considered, selecting the appropriate upsetting length and increasing the width of the high-temperature plastic zone are beneficial for the extrusion of harmful impurities. Moreover, using variable speed upsetting or increasing the speed of the early upsetting facilitates the extrusion of impurities. However, the impurities in the deeper areas of the rail are difficult to move, and they easily form gray spot defects if the oxide inclusions remain.