Multiparametric Maximum Likelihood Estimation

Definition

Multiparameter MLE

Let $X_1,X_2,\dots ,X_n$ be Random Sample with PDF $f(x|\mathbf{\theta })$, where $\mathbf{\theta }=({\theta }_1,{\theta }_2,\dots ,{\theta }_p{)}^{⊺}\in \Omega \subset R^p$, then the MLE ${\hat{\mathbf{\theta }}}_{\text{MLE}}$ of $\mathbf{\theta }$ is estimated as ${\hat{\mathbf{\theta }}}_{\text{MLE}}=\underset{\mathbf{\theta }}{\mathrm{argmax}}L(\mathbf{\theta }|\mathbf{x})$

Properties

Multiparameter Asymptotic Normality

Under the R0 ~ R5 Regularity Conditions, let $X_1,X_2,\dots ,X_n$ be Random Sample with PDF $f(x|\mathbf{\theta })$, where $\mathbf{\theta }=({\theta }_1,{\theta }_2,\dots ,{\theta }_p{)}^{⊺}\in \Omega \subset R^p$, then

• $\frac{\partial l(\mathbf{\theta })}{\partial \mathbf{\theta }}=0$ has solution ${\hat{\mathbf{\theta }}}_n$ such that ${\hat{\mathbf{\theta }}}_n\stackrel{P}{\to }\mathbf{\theta }$
• $\sqrt{n}({\hat{\mathbf{\theta }}}_n-\mathbf{\theta })\stackrel{D}{\to }{N}_p(0,I^{-1}(\mathbf{\theta }))$
• $\forall j\in \{1,2,\dots ,p\},\sqrt{n}({\hat{\theta }}_{nj}-{\theta }_j)\stackrel{D}{\to }N(0,I^{-1}(\mathbf{\theta }{)}_{jj})$

Delta Method for Multiparameter MLE

Multiparameter Rao-Cramer Lower Bound

Let $X_1,X_2,\dots ,X_n$ be Random Sample with PDF $f(x|\mathbf{\theta })$, $Y_j=u(X_1,X_2,\dots ,X_n)$ be an unbiased estimator of ${\theta }_j$, then $E(Y_j)=k(\theta )={\theta }_j$. Thus, by a Rao-Cramer Lower Bound, $\mathrm{Var}(Y_j)\ge \frac{1}{n}{I}^{-1}(\mathbf{\theta }{)}_{jj}$ where $I^{-1}(\mathbf{\theta }{)}_{jj}$ is $j$-th diagonal element of $I^{-1}(\mathbf{\theta })$

If $=$ holds, then $Y_j$ is called an efficient estimator of ${\theta }_j$

In a location-scale model, the information of the parameters do not depend on the locale parameter.