Often neither the exact density nor the exact cumulative distribution function (c.d.f.) of a statistic of interest is available in the statistics and econometrics literature (e.g., the maximum likelihood estimator of the autocorrelation coefficient in a simple Gaussian AR(1) model with zero start-up value). In other cases the exact c.d.f. of a statistic of interest is very complicated despite the statistic being “simple” (e.g., the circular serial correlation coefficient, or a quadratic form of a vector uniformly distributed over the unit n-sphere). The first part of the paper tries to explain why this is the case by studying the analytic properties of the c.d.f. of a statistic under very general assumptions. Differential geometric considerations show that there can be points where the c.d.f. of a given statistic is not analytic, and such points do not depend on the parameters of the model but only on the properties of the statistic itself. The second part of the paper derives the exact c.d.f. of a ratio of quadratic forms in normal variables, and for the first time a closed form solution is found. These results are then specialized to the maximum likelihood estimator of the autoregressive parameter in a Gaussian AR(1) model with zero start-up value, which is shown to have precisely those properties highlighted in the first part of the paper.