A key feature of gauge theories is the presence of non-physical degrees of freedom whose time evolution is not determined by equations of motion. It seems natural to eliminate such degrees of freedom from the theory and obtain an effective dynamics only for physical variables. In this chapter this problem is studied in detail. There are two ways to achieve this goal, invariant and non-invariant. In the invariant approach, only gauge-invariant variables are used, while the non-invariant route is associated with the use of quantities that are not gauge invariant. However, it only appears non-invariant, similarly to the use of local coordinates on a manifold to compute coordinate-invariant quantities, e.g. curvature or geodesic length. Both approaches describe the very same theory and, therefore, must be equivalent. Unfortunately, there is a shortage of rigorous studies to establish the equivalence of the invariant and non-invariant approaches for general gauge systems. For that reason the discussion is limited to concrete models. The analysis of the physical phase space structure is in the focus of the discussion since this problem has been studied only recently and is not discussed in the monograph literature and textbooks on gauge theories. It appears that for a large class of gauge theories the physical phase space may have a non-trivial structure, i.e. different from the conventional symplectic plane for each physical degree of freedom. This leads to several new phenomena in classical as well as quantum dynamics [183] (see reviews [26, 195]).