Bayes Classifier

Definition

${\hat{f}}_{\text{Bayes}}(x)=\underset{1\le k\le K}{\mathrm{argmax}}P(Y=k|X=x)=\underset{1\le k\le K}{\mathrm{argmax}}{p}_k(x){\pi }_k$ where $p_k(x)=P(x|Y=k)$ is conditional PDF of $x$ given $Y=k$ and ${\pi }_k=P(Y=k)$ is prior

Properties

Optimality of Bayes Rule

Bayes classifier ${\hat{f}}_{\text{Bayes}}(x)$ minimizes the prediction risk over all classifiers

Facts

For a given $x$, the Bayes classifier ${\hat{f}}_{\text{Bayes}}(x)$ is deterministic, i.e. it does not depend on the training sample. It can not be used in practice since $P(Y=k|X=x)$ is unknown.

Bayes classifier ${\hat{f}}_{\text{Bayes}}(x)$ maximizes the posterior pdf.