Lack of Fit Test

Definition

Consider a Multiple Linear Regression model $\mathbf{y}=\mathbf{X}\mathbf{\beta }+\mathbf{ϵ}$ We want to test the linearity of the model $H_0:E(Y_i)={\mathbf{X}}_i^{⊺}\mathbf{\beta },i=1,2,\dots ,n$

Assume that $Y_{il}(i=1,2,\dots ,m;l=1,2,\dots ,n_i)$ is the $l$-th replication at $x_i$, then The $SSE$ is decomposed to the sum of the pure error sum of squares $SSPE$ and the lack-of-fit sum of squares $SSLF$. $\underset{SSE}{\underbrace{\sum _{i=1}^m\sum _{l=1}^{n_i}(Y_{il}-{\hat{Y}}_i{)}^2}}=\underset{SSPE}{\underbrace{\sum _{i=1}^m\sum _{l=1}^{n_i}(Y_{il}-{\bar{Y}}_i{)}^2}}+\underset{SSLF}{\underbrace{\sum _{i=1}^m\sum _{l=1}^{n_i}({\bar{Y}}_i-{\hat{Y}}_i{)}^2}}$

The degree of freedoms are ${\text{df}}_{SSE}=n-p$, ${\text{df}}_{SSPE}=n-m$, and ${\text{df}}_{SSLF}=m-p$, where $n:=\sum _{i=1}^m{n}_i$

Also, the mean squares are obtained by dividing the sum of squares by its degree of freedom $M{S}_{PE}=\frac{{\sum }_{i=1}^m{\sum }_{l=1}^{n_i}(Y_{il}-{\bar{Y}}_i{)}^2}{n-m},M{S}_{LF}=\frac{{\sum }_{i=1}^m{\sum }_{l=1}^{n_i}({\bar{Y}}_i-{\hat{Y}}_i{)}^2}{m-p}$

Under $H_0$, the $M{S}_{LF}$ is an unbiased estimator of ${\sigma }^2$, and $M{S}_{PE}$ is always an unbiased estimator of ${\sigma }^2$, regardless of $H_0$. Therefore, we can test the hypothesis by comparing ratios of both estimators.

$\frac{S{S}_{LF}}{{\sigma }^2}\sim {\chi }^2(n-p)$ and $\frac{S{S}_{PE}}{{\sigma }^2}={\chi }^2(n-m)$. Therefore, $F_{L}F=\frac{M{S}_{LF}}{M{S}_{PE}}\sim F(m-p,n-m)$

Hence, we reject $H_0$ if $F>F_{\alpha }(m-p,n-m)$

Facts

The lack of fit test uses data’s replication, Therefore, there should be more than one data (replication) for some $x_i$‘s.