We review the current status of searches for high redshift galaxies and quasars, together with their derived luminosity function, star formation history and relative contribution to the UV ionizing background. These searches have been carried out in the optical and near-infrared with the HST for very deep pencil beam surveys and ground-based telescopes for deep large area surveys. The peak of galaxy and black hole formation activity occurs at $z\sim 2$-3, with a strong decrease in UV luminosity density in the redshift interval $z\sim 3$ to 6. This decrease seems to extend beyond $z\sim 6$ for luminous galaxies, as there are only a few $z\sim 7$-8 secure galaxy candidates. The star formation rate at $z\sim 6$ is dominated by 0.1 L$^{\star}$ galaxies. The $z \gtrsim 5$ samples for both types of sources are too small to constrain the faint end of their luminosity function or even its knee. Consequently, predictions of the number density of $z \gtrsim 7$ galaxies is highly uncertain; their search with JWST and ELTs should cover areas of at least $\sim$1 deg$^2$ and will require long campaigns (${\gtrsim} 1$ month). Faint quasars of small black hole mass should also be detected in these surveys. Searches for $z\sim10$ rare, bright quasars and higher $z$ transient sources, gamma-ray bursts and population III SNe, should be undertaken with dedicated facilities. The physical properties of high $z$ galaxies and the metal-enrichment of the intergalactic medium at early epochs will be determined by high-resolution spectroscopic observations with ELTs requiring substantial exposure times (${\sim} 100$ hr).