The General Linear Test

Definition

Consider a Multiple Linear Regression model $\mathbf{y}=\mathbf{X}\mathbf{\beta }+\mathbf{ϵ}$ The general linear test can be written as $H_0:\mathbf{C}\mathbf{\beta }=\mathbf{m},H_1:\mathbf{C}\mathbf{\beta }\ne \mathbf{m}$ where $\mathbf{C}$ is $q\times p$ matrix with rank $q$ and $\mathbf{m}$ is $q\times 1$ vector.

e.g. for the hypothesis $H_0:{\beta }_1-{\beta }_2=0,{\beta }_3+2{\beta }_4=3$, $C=\left[\begin{array}{ccccc}0& 1& -1& 0& 0\\ 0& 0& 0& 1& 2\end{array}\right],\mathbf{m}=[0,3{]}^{⊺}$

${\text{df}}_R=n-(p-q)$ and ${\text{df}}_F=(n-p)$. Therefore, $F=\frac{SSE(R)-SSE(F)}{q\cdot MSE(F)}\sim F(q,n-p)$ where $MSE(F)=SSE(F)/{\text{df}}_F$

Hence, we reject $H_0$ if $F>F_{\alpha }(q,n-p)$

Method of Lagrange Multiplier

By the method of Lagrange multiplier, $SSE(R)-SSE(F)=(\mathbf{C}\hat{\mathbf{\beta }}-\mathbf{m}{)}^{⊺}[\mathbf{C}({\mathbf{X}}^{⊺}\mathbf{X}{)}^{-1}{\mathbf{C}}^{⊺}{]}^{-1}(\mathbf{C}\hat{\mathbf{\beta }}-\mathbf{m})$ Therefore, we can calculate the F-statistic without fitting the reduced model $F=\frac{(\mathbf{C}\hat{\mathbf{\beta }}-\mathbf{m}{)}^{⊺}[\mathbf{C}({\mathbf{X}}^{⊺}\mathbf{X}{)}^{-1}{\mathbf{C}}^{⊺}{]}^{-1}(\mathbf{C}\hat{\mathbf{\beta }}-\mathbf{m})}{q\cdot MSE(F)}\sim F(q,n-p)$