Linear Discriminant Analysis

Definition

$\underset{1\le k\le K}{\mathrm{argmax}}{\hat{f}}_k(x)=\underset{1\le k\le K}{\mathrm{argmax}}\left[\ln {\hat{\pi }}_k+x^{⊺}{\hat{\Sigma }}^{-1}{\hat{\mu }}_k-\frac{1}{2}{\hat{\mu }}_k^{⊺}{\hat{\Sigma }}^{-1}{\hat{\mu }}_k\right]$ where ${\hat{\pi }}_k=\frac{n_k}{n}$ is the sample ratio, ${\hat{\mu }}_k=\frac{1}{n_k}\sum _{i:Y_i=k}{X}_i$ is the sample mean, and $\hat{\Sigma }=\frac{1}{n-K}\sum _{k=1}^K\sum _{i:Y_i=k}(X_i-{\hat{\mu }}_k)(X_i-{\hat{\mu }}_k{)}^{⊺}$ is the sample variance covariance matrix

LDA is a Bayes Classifier with an assumption that $p_k(x)\sim N_p({\mu }_k,\Sigma )$

Facts

LDA assume homogeneity of variance-covariance among classes. Quadratic Discriminant Analysis may be used when the covariances are not equal.