Distribution of Regression Coefficient

Definition

Distribution of Regression Coefficient

Assume that $\mathbf{y}\sim N_n(\mathbf{X}\mathbf{\beta },{\mathbf{I}}_n{\sigma }^2)$ The Least Squares Estimator $\hat{\mathbf{\beta }}=({\mathbf{X}}^{⊺}\mathbf{X}{)}^{-1}{\mathbf{X}}^{⊺}\mathbf{y}$ also follows Normal Distribution. $\hat{\mathbf{\beta }}\sim N_p(\mathbf{\beta },({\mathbf{X}}^{⊺}\mathbf{X}{)}^{-1}{\sigma }^2)$ where $p-1$ is the number of explanatory variables.

Marginal Distribution of Regression Coefficient

The marginal distribution of the Multivariate Normal Distribution is univariate Normal Distribution ${\hat{\beta }}_j=N({\beta }_j,c_{j+1}{\sigma }^2)$ where $c_{j+1}=\mathrm{diag}[({\mathbf{X}}^{⊺}\mathbf{X}{)}^{-1}{]}_{j+1}$