The young (12+8−4 Myr) and nearby (19.44±0.05 pc) star β Pictoris is considered one of the best laboratories for the study of early phases of planetary systems formation since the identification of an extended debris disk surrounding the star in 1984. In 2009, we imaged at 3.8 μm with NaCo at VLT a gas giant planet around β Pictoris, roughly along the disk mid-plane, with a semi-major axis between 8 and 14 AU. We present here the first images of the planet in the J (1.265 μm), H (1.66 μm), and M' (4.78 μm) bands obtained between 2011 and 2012. We used these data to build the 1-5 μm spectral energy distribution (SED) of the companion, and to consolidate previous semi-major axis (8-10 AU) estimates. We compared the SED to seven atmospheric models to derive Teff = 1700 ± 100 K. We used the temperature and the luminosity of β Pictoris b to estimate new masses for the companion. We compared these masses to independent constraints set by the orbital parameters and the radial velocities and use them to discuss the formation history of the object.