Wasserstein Distance

Definition

Wasserstein distance is the minimum cost required to transform one distribution to another.

Assume that two Probability Distributions $P$ and $Q$ are given. Then the Wasserstein distance between $P$ and $Q$ is defined as $W(P,Q)={\inf }_{\gamma \in \Gamma (P,Q)}{E}_{(x,y)\sim \gamma }[|x-y|]={\int }_0^1\left|F_P^{-1}(y)-F_Q^{-1}(y)\right|dy$ where $\Gamma (P,Q)$ is the set of all joint distributions $\gamma (x,y)$ with marginals $P$ and $Q$ respectively. Intuitively $\gamma (x,y)$ indicates how much mass must be transported from $x$ to $y$ in order to transform $P$ into $Q$.

p-Wasserstein Metric

$W_p(P,Q)={\inf }_{\gamma \in \Gamma (P,Q)}(E_{(x,y)\sim \gamma }[|x-y{|}^p]{)}^{1/p}={\left({\int }_0^1{\left|F_P^{-1}(y)-F_Q^{-1}(y)\right|}^{p}dy\right)}^{1/p}$

Facts

Let $\{P_n{\}}_{n\in \mathbb{N}}$ be a sequence of distributions. Then, $D_{KL}(P_n||P)\to 0⟹D_{JS}(P_n,P)\to 0⟺\delta (P_n,P)\to 0⟹W(P_n,P)\to 0⟺P_n\stackrel{D}{\to }P$ The convergence of the KL-Divergence to zero implies that the JS-Divergence also converges to zero. The convergence of the JS-Divergence to zero is equivalent to the convergence of the Total Variation Distance to zero. The convergence of the Total Variation Distance to zero implies that the Wasserstein Distance also converges to zero. The convergence of the Wasserstein Distance to zero is equivalent to the Convergence in Distribution of the sequence.

Link to original