Multinomial Logistic Regression

Definition

Multinomial logistic regression is used for polychotomous (multi-class) data.

Suppose the number of classes is $K$ $\ln \frac{P(Y=k|X=x)}{P(Y=K|X=x)}={\mathbf{\beta }}_k^{⊺}x,k=1,\dots ,K-1$ The probability is calculated as ${\pi }_k=P(Y=k|X=\mathbf{x})=\{\begin{array}{ll}\frac{\exp ({\mathbf{\beta }}_k^{⊺}\mathbf{x})}{1+{\sum }_{l=1}^{K-1}\exp ({\mathbf{\beta }}_l^{⊺}\mathbf{x})}& k=1,\dots ,K-1\\ \frac{1}{1+{\sum }_{l=1}^{K-1}\exp ({\mathbf{\beta }}_l^{⊺}\mathbf{x})}& k=K\end{array}$ $\underset{1\le k\le K}{\mathrm{argmax}}{\hat{f}}_k(\mathbf{x})=\underset{1\le k\le K}{\mathrm{argmax}}\hat{P}(Y=k,X=\mathbf{x})$