Brief Overview of Generalized Linear Models

The normal model, also referred to as linear regression, was discussed in Chapter 4.The characteristic assumption of the normal model is that the data being modeled is normally distributed. A normal or Gaussian distribution has two parameters, a location or mean parameter μ and a variance or scale parameter σ2. In most frequency-based linear models the variance is not estimated but is regarded as having a fixed or constant value. A full maximum likelihood estimation of a normal model, however, does estimate both the mean and variance parameters. We found that both parameters are estimated in a Bayesian normal model. It is important to remember that analysts employing a Bayesian model nearly always estimate all the parameters associated with the probability function considered to underlie a model.

Generalized linear models (GLMs) are fundamental to much contemporary statistical modeling. The normal or linear model examined in the previous chapter is also a GLM and rests at the foundation of this class of models. However, the true power of GLMs rests in the extensions that can be drawn from it. The results are binary, binomial or proportional, multinomial, count, gamma, and inverse Gaussian models, mixtures of these, and panel models for the above to account for clustering, nesting, time series, survival, and longitudinal effects. Other models can be developed from within this framework, so it is worth taking time to outline what is involved in GLM modeling and to see how it can be extended. Bayesian models can be derived from these distributions, and combinations of distributions, in order to obtain a posterior distribution that most appropriately represents the data being modeled. Despite the ubiquitous implementation of GLMs in general statistical applications, there have been only a handful of astronomical studies applying GLM techniques such as logistic regression (e.g. de Souza et al., 2015a, 2016; Lansbury et al., 2014; Raichoor and Andreon, 2014), Poisson regression (e.g. Andreon and Hurn, 2010), gamma regression (Elliott et al., 2015), and negative binomial (NB) regression (de Souza et al., 2015b).

The generalized linear model approach was developed by statisticians John Nelder and Robert Wedderburn in 1972 while working together at the Rothamsted Experimental Station in the UK. Two years later they developed a software application for estimating GLMs called Generalized Linear Interactive Models (GLIM), which was used by statisticians worldwide until it was discontinued in 1994.