Decomposition of Sum of Squares

Definition

Simple Linear Regression Case

Consider a Simple Linear Regression model $Y_i={\beta }_0+{\beta }_1{X}_i+{ϵ}_i$ The total sum of squares $SST={\sum }_{i=1}^n(Y_i-\bar{Y}{)}^2$ is expressed by the sum of the unexplained variation $SSE$ and the explained variation $SSR$ $SST=SSE+SSR$ where

• $SST={\sum }_{i=1}^n(Y_i-\bar{Y}{)}^2$ is the total sum of squares and has the degree of freedom $(n-1)$,
• $SSE={\sum }_{i=1}^n(Y_i-\hat{Y}{)}^2$ is the error sum of squares and has the degree of freedom $(n-2)$, and
• $SSR={\sum }_{i=1}^n({\hat{Y}}_i-\bar{Y}{)}^2$ is the regression sum of squares and has the degree of freedom $1$

Multiple Linear Regression Case

Consider a Multiple Linear Regression model $\mathbf{y}=\mathbf{X}\mathbf{\beta }+\mathbf{ϵ}$

• $SST={\sum }_{i=1}^n(Y_i-\bar{Y}{)}^2$ has the degree of freedom $(n-1)$,
• $SSE={\sum }_{i=1}^n(Y_i-\hat{Y}{)}^2$ has the degree of freedom $(n-p)$, and
• $SSR={\sum }_{i=1}^n({\hat{Y}}_i-\bar{Y}{)}^2$ has the degree of freedom $p-1$

Facts

$SST/{\sigma }^2\sim {\chi }^2\left((n-1),\frac{1}{2{\sigma }^2}{\mathbf{\beta }}^{⊺}{\mathbf{X}}^{⊺}(I-\frac{1}{n}J)\mathbf{X}\mathbf{\beta }\right)$ $SSE/{\sigma }^2\sim {\chi }^2(n-p)$ $SSR/{\sigma }^2\sim {\chi }^2\left((p-1),\frac{1}{2{\sigma }^2}{\mathbf{\beta }}^{⊺}{\mathbf{X}}^{⊺}(I-\frac{1}{n}J)\mathbf{X}\mathbf{\beta }\right)$