Millimeter emission is known to be a sensitive diagnostic of temperature and density in the solar chromosphere. In this work we use millimeter wave data to distinguish between various atmospheric models of sunspots, whose temperature structure in the upper photosphere and chromosphere has been the source of some controversy. From mm brightness simulations we expect a radio umbra to change its appearance from dark to bright (compared to the Quiet Sun) at a given wavelength in the millimeter spectrum (depending on the exact temperature in the model used). Thereby the millimeter brightness observed above an umbra at several wavelengths imposes strong constraints on temperature and density stratification of the sunspot atmosphere, in particular on the location and depth of the temperature minimum and the location of the transition region. Current mm/submm observational data suggest that brightness observed at short wavelengths is unexpectedly low compared to the most widely used sunspot models such as of Maltby et al. (1986). A successful model that is in agreement with millimeter umbral brightness should have an extended and deep temperature minimum (below 3000 K), such as in the models of Severino et al. (1994). However, we are not able to resolve the umbra cleanly with the presently available observations and better resolution as well as better wavelength coverage are needed for accurate diagnostics of umbral brightness at millimeter wavelengths. This adds one more scientific objective for the Atacama Large Millimeter/Submillimeter Array (ALMA).