Intense bursts of solar radio emission were first recognized by Second World War British and Australian coastal radar systems as well as by German and Japanese radar systems. Due to wartime security, these discoveries were not declassified until after the end of hostilities but, before declassification, Grote Reber, working alone in his mother’s backyard, reported receiving surprising strong radio emission from the Sun, well in excess of the expected emission from the 5,000 K solar surface. In 1946, while demonstrating his equipment to government representatives, Reber rediscovered solar radio storms when his chart recorder went off scale. Following World War II, with rapidly improving instrumentation, the Sun became a major target in the emerging field of radio astronomy. Observations with instruments of increasing sophistication have traced the complex time, frequency, and spatial dependence of the solar radio emission which corresponded to a wide variety of emission mechanisms. Later, following a false start due to using incorrect positions, radio emission was also detected from a variety of stars in our Galaxy, opening up the new field of stellar radio astronomy.

The history of radio astronomy has been a series of discoveries, mostly serendipitous, using a new instrument, or using an old instrument in a new unintended way. Theoretical predictions have had little influence, and in some cases actually delayed the discovery by discouraging observers. Many of the key transformational discoveries were made while investigating other areas of astronomy; others came as a result of commercial and military pursuits unrelated to astronomy. We discuss how the transformational serendipitous discoveries in radio astronomy depended on luck, age, education, and the institutional affiliation of the scientists involved, and we comment on the effect of peer review in the selection of research grants, observing time, and the funding of new telescopes, and speculate on its constraint to new discoveries. We discuss the decrease in the rate of new discoveries since the Golden Years of the 1960s and 1970s and the evolution of radio astronomy to a big science user oriented discipline. We conclude with a discussion of the impact of computers in radio astronomy and speculations on the potential for future discoveries in radio astronomy – the unknown unknowns.

The realization that radio astronomers could detect radio galaxies that were well beyond the limits of even the most powerful optical telescopes suggested that radio observations might be able to distinguish between the two competing cosmological models. The commonly accepted big-bang model required that, since the Universe continued to evolve with time, so the distant (younger) Universe should appear different than the nearby modern Universe. By contrast, the steady-state theory required that the Universe is, and always was, everywhere the same, so distant galaxies should look the same as more nearby galaxies. An intense controversy developed between radio astronomers in Sydney, Australia and Cambridge, UK over the distribution of radio sources and their implication for theories of cosmology. The Australian radio astronomers, who had better data than the Cambridge research workers, found no evidence of cosmic evolution. The Cambridge group, led by Martin Ryle, misunderstood the effects of their instrumental errors and used an incorrect analysis – but got the right answer, arguing that the Universe is evolving with time, contrary to the expectations of the steady state-theory.

Some celestial objects, later recognized as quasars, were catalogued back in 1887, and their extragalactic nature was discussed as early as 1960. However, the large measured redshift of 3C 48 was rejected, largely because it implied an unrealistically high radio and optical luminosity. Instead it was assumed to be a relatively nearby, less luminous galactic radio star. Following the 1962 observations of lunar occultations of the strong radio source 3C 273 at the Parkes radio telescope and the subsequent identification with an apparent stellar object, Martin Schmidt recognized that 3C 273 had an unmistakable redshift of 0.16. Due to an error in the calculation of the radio position, the occultation position actually played no direct role in the identification of 3C 273, although it was the existence of a claimed accurate occultation position that motivated Schmidt’s 200 inch telescope investigation and his determination of the redshift. Later radio and optical measurements quickly led to the identification of other quasars with increasingly large redshifts, although the nature of the quasar redshifts remained controversial for decades.

After radio surveys of the sky uncovered a variety of discrete radio sources, there was an intense debate as to whether the radio emission originated in nearby “radio stars,” or were powerful sources located in distant galaxies? In Australia, John Bolton and Gordon Stanley discovered radio emission from two known galaxies. However, unwilling to accept the implied powerful radio emission if the sources were so distant, they instead erroneously reported that the optical counterparts to the radio sources were nearby Galactic nebulosities and not remote galaxies. Later, another Australian scientist identified the bright Cygnus A radio source with a faint galaxy and drew this identification to the attention of Mt. Wilson and Palomar astronomers, who initially either ignored or rejected the identification as being unrealistic. But, a few years later, they independently reidentified the Cygnus A radio sources, firmly establishing the nature of powerful radio galaxies and leading to wide-ranging speculation about the source of the apparent huge energy needed to power the giant radio lobes that typically extended hundreds of thousands of light years from the host galaxy.

Surprisingly, radio observations of the planets also turned up unexpected discoveries. While testing their transit radio array using the Crab Nebula as a reference source, two scientists at the Carnegie Institution Department of Terrestrial Magnetism noticed a strange variable signal that repeated each night. First suspecting that it was ignition noise from a nearby farm vehicle, they later realized that they were detecting radio emission from powerful electrical storms on Jupiter, which was at the same declination as the Crab Nebula as it passed through their fixed telescope beam. Mercury, long thought to have one side bathed in eternal daylight, was found to be rotating. Radio observations revealed the greenhouse effect on Venus, causing surface temperatures to reach over 600 degrees Celsius, and detected intense radiation belts around Jupiter, analogous to the Earth’s van Allen belts. The other giant planets were all found to be warmer than can be explained by solar heating alone. Precise pulsar timing measurements disclosed the first known extrasolar planetary system, a precursor to the thousands of extrasolar planets later discovered by ground and space based optical studies.

One important area where radio astronomers confirmed theoretical predictions was in tests of General Relativity. Radio interferometer measurements made during the 1970s were able to confirm Einstein’s prediction of the gravitational bending of light to an accuracy better than 1 percent, or an order of magnitude better than the controversial classical optical tests made during the time of a solar eclipse. In 1965, MIT Professor Irwin Shapiro suggested and subsequently confirmed a new fourth test of General Relativity resulting from the excess delay of the reflected radar signal from a planet as the signal passes close to the Sun. Radio observations have also found Einstein’s “gravitational lensing” by which a massive cluster of galaxies can form multiple radio images of a background galaxy or quasar. Observations of small periodic deviations in the time of arrival of pulsar pulses at the Arecibo Observatory led Princeton University graduate student Russell Hulse and his supervisor Joe Taylor to the 1993 Nobel Prize in Physics for the first experimental evidence for the predicted existence of gravitational radiation.

The existence of a Cosmic Microwave Background was theoretically predicted by George Gamow and his associates, but played no role in the accidental discovery of the 2.7 degree cosmic microwave background radiation by Penzias and Wilson while they were testing a new type of satellite communications antenna at the Bell Telephone Laboratories in Holmdel, NJ. An earlier measurement of the cosmic microwave background at Bell Labs went unnoticed except by Russian scientists, who misunderstood the paper to be reporting a negative result. Meanwhile, not far away, Robert Dicke and his colleagues at Princeton University were building a radiometer to verify Dicke’s prediction that it might be possible to detect the microwave remnants of the big-bang. But they were beaten by Penzias and Wilson’s serendipitous Nobel Prize winning discovery that led to the final demise of the steady-state theory. An even earlier measurement of optical absorption lines by interstellar cyanogen gave the first clues to the existence of a cosmic background radiation, but its meaning was not recognized until after the 1965 experimental discovery of the microwave background at Bell Labs.

While trying to discover additional quasars, Cambridge University graduate student, Jocelyn Bell noticed a peculiar looking signal that turned up around the same time each night. With determined curiosity and drive, Bell realized that the peculiar signal was pulsing with a 1.3 second repetition rate. The remarkable discovery of pulsars confirmed the theoretical prediction of the existence of neutron stars, but the prediction played no role in Bell’s serendipitous detection of the first pulsars. Meanwhile, at a remote Alaska radar DEW Line station, US Air Force officer Charles Schisler observed pulses even when his radar system was not transmitting. Following an off-duty investigation at the Fairbanks library, he realized that these pulses had a cosmic origin and were not coming from approaching Soviet missiles, but his independent discovery of pulsars remained classified for decades. Later, while searching for new pulsars in old data from the Parkes radio telescope, West Virginia University radio astronomer Duncan Lorimer discovered a new phenomenon, known as fast radio bursts, which were confused with a similar bursting type radiation from a microwave oven.

The introductory chapter places radio astronomy in the context of the broader astronomical environment. The transformational discoveries made by radio astronomy and the circumstances surrounding these discoveries are summarized with an emphasis on the role of serendipity and its impact on science.

In a talk given during the German occupation of the Netherlands, Henk van der Hulst discussed possible 21 cm radio emission from interstellar hydrogen atoms but pessimistically concluded that “the existence of the line remains speculative.” Nearly 20 years later, Harvard University PhD student Harold (Doc) Ewen surprisingly detected the 21 cm hydrogen line using a simple horn antenna sticking out the window of his laboratory and a novel frequency switching radiometer. van de Hulst had also calculated the possibility of detecting radio recombination lines from highly excited galactic hydrogen, but overestimated the effect of line broadening. Although he concluded that radio recombination lines are “unobservable,” they were subsequently detected in the USSR and the US. Observations of surprisingly strong radio emission from hydroxyl and water vapor were understood to be due to interstellar masers, which could have been detected much earlier if anyone had thought to look in the right place. Later discoveries of interstellar formaldehyde and carbon monoxide opened the door to a new and highly competitive field of astrophysics – molecular radio spectroscopy.