Likelihood Ratio Test

Definition

Let $X_1,X_2,\dots ,X_n$ be Random Sample with PDF $f(x|\mathbf{\theta })$ where $\mathbf{\theta }\in \Omega$, $H_0:\mathbf{\theta }\in \omega ,H_1:\mathbf{\theta }\in \Omega \cap {\omega }^{\complement }$, where $\omega \subset \Omega$. $\Lambda =\frac{L(\hat{\omega })}{L(\hat{\Omega })}$ is called a likelihood ratio, where $\hat{\omega },\hat{\Omega }$ are MLE under $\omega$ and $\Omega$ respectively. If $H_0$ is true, then $\Lambda$ will be close to $1$, while if $H_0$ is not true, then the $\Lambda$ will be close to $0$

Therefore, LRT rejects $H_0$ when $\Lambda \le c$, where the $c$ is determined by the level alpha condition. $\alpha =P_{\omega }(\Lambda \le c)$

By the Wilks’ Theorem, the likelihood ratio test Statistic for the hypothesis $H_0:\theta ={\theta }_0,H_1:\theta \ne {\theta }_0$ is given by ${\chi }_L^2={\lambda }_{LR}:=-2\ln \Lambda =-2\ln \frac{L({\theta }_0)}{L(\hat{\theta })}\stackrel{D}{\to }{\chi }^2(1)$

Visualization

The comparison of the Likelihood Ratio Test, Wald Test, and Score Test