Multiple Linear Regression

Definition

$Y_i={\beta }_0+{\beta }_1{X}_{i1}+{\beta }_2{X}_{i2}+\cdots +{\beta }_{p-1}{X}_{i,p-1}+{ϵ}_i,i=1,2,\dots ,n$ where ${ϵ}_i$‘s are i.i.d. error terms, with $E({ϵ}_i)=0$ and $\mathrm{Var}({ϵ}_i)={\sigma }^2$

Matrix Notations

Let $\mathbf{\beta }=({\beta }_0,{\beta }_1,{\beta }_2,\dots ,{\beta }_{p-1}{)}^{⊺}$ and ${\mathbf{x}}_i=(1,X_{i1},X_{i2},\dots ,X_{i,p-1}{)}^{⊺}$, then the regression model can be express the model as $Y_i={\mathbf{x}}_i^{⊺}\mathbf{\beta }+{ϵ}_i,i=1,2,\dots ,n$

Let $\mathbf{y}=[Y_1,Y_2,\dots ,Y_n{]}^{⊺}$, $\mathbf{X}=[{\mathbf{x}}_1^{⊺},{\mathbf{x}}_2^{⊺},\dots ,{\mathbf{x}}_n^{⊺}{]}^{⊺}$, $\mathbf{ϵ}=[{ϵ}_1,{ϵ}_2,\dots ,{ϵ}_n{]}^{⊺}$, then the regression model can be express as $\mathbf{y}=\mathbf{X}\mathbf{\beta }+\mathbf{ϵ}$ where $E[\mathbf{ϵ}]=0$, and $\mathrm{Cov}[\mathbf{ϵ}]={\sigma }^2{I}_n$