With three examples of forward modeling of spectral intensity formation we illustrate some of the difficulties encountered in the use of spectra for the determination of physical properties of highly structured magnetized plasmas in general, and the Sun in particular. We demonstrate that the average spectrum of a structured atmosphere cannot be use for an accurate determination of the atmosphere’s average properties, show that why it is difficult to measure the chromospheric magnetic vector field from Zeeman polarization, and make clear from forward modeling of spectral lines that it will be hard to fully recover the physical structure of a sunspot.