Cross Entropy

Definition

$H(P,Q)=-E_{x\sim p(x)}[\log q(x)]=-{\int }_{x}p(x)\log q(x)dx$

The cross entropy between two probability distributions $p$ and $q$ quantifies the difference of them