Weighted Least Squares

Definition

Weighted least squares (WLS) is a generalization of Least Square to cope with Heteroskedasticity.

We assume that $\mathrm{Cov}(\mathbf{ϵ})={\sigma }^2{\mathbf{W}}^{-1}$ where $\mathbf{W}=\mathrm{diag}(w_1,w_2,\dots ,w_n)$ The weight matrix is positive definite, so there exists a non-singular matrix $\mathbf{P}:=W^{-1/2}$. Consider a transformed model $\mathbf{Py}=\mathbf{PX}\mathbf{\beta }+\mathbf{P}\mathbf{ϵ}$ Now, the new error term $\mathbf{P}\mathbf{ϵ}$ is i.i.d. $\mathrm{Cov}(\mathbf{P}\mathbf{ϵ})={\sigma }^2\mathbf{I}$ And the weighted least squares estimator is obtained by ${\hat{\mathbf{\beta }}}_{WLS}=({\mathbf{X}}^{⊺}\mathbf{WX}{)}^{-1}{\mathbf{X}}^{⊺}\mathbf{Wy}$