Generalized Linear Model

Definition

A generalized linear model (GLM) is a generalization of linear regression model.

The GLM consists of three elements.

1. Response variable: The i.i.d. response variables $Y_i$ follow Exponential Family.
2. Linear predictor: $\mathbf{\eta }:={\mathbf{X}}^{⊺}\mathbf{\beta }$ is called a linear predictor
3. Link function: There exists a link function satisfies $g(\mu )=\mathbf{\eta }$, where $g$ is assumed to be monotone and differentiable.

Link Functions

Link functions for binomial response.

• Logit: $\eta =\ln \left(\frac{\mu }{1-\mu }\right)$
• Probit: $\eta ={\Phi }^{-1}(\mu )$ where $\Phi$ is a CDF of Normal Distribution
• Complementary log-log: $\eta =\ln [-\ln (1-\mu )]$

For an Exponential Family, the link function satisfying $\theta =\eta$ is called natural link.

Facts

The parameter $\mathbf{\beta }$ is estimated by MLE. Since the score function is non-linear, it can’t have explicit solution, so we use Newton–Raphson method or Fisher’s Scoring Method.