Kernel Density Estimation

Definition

${\hat{f}}_h(x)=\frac{1}{n}\sum _{i=1}^n{K}_h(x-X_i)=\frac{1}{nh}\sum _{i=1}^{n}K\left(\frac{x-X_i}{h}\right)$ where $K\text{s.t.}{\int }_{-\infty }^{\infty }K(t)dt=1,{\int }_{-\infty }^{\infty }{K}^2(t)dt<\infty$ is the kernel, $K_h(x):=\frac{1}{h}K(\frac{x}{h})$ is the scaled kernel, and $h$ is a smoothing parameter (bandwidth, or window width)

KDE is a non-parametric method to estimate PDF of a random variable based on kernel and wrights.

Multidimensional KDE

${\hat{f}}_H(x)=\frac{1}{n}\sum _{i=1}^n{K}_H(\mathbf{x}-{\mathbf{x}}_i)=\frac{1}{n}|H{|}^{-1}\sum _{i=1}^{n}K(H^{-1}(\mathbf{x}-{\mathbf{x}}_i))$ where $K\text{s.t.}{\int }_{-\infty }^{\infty }K(\mathbf{u})d\mathbf{u}=1,K\ge 0$ is the kernel, $K_H(x):=|H{|}^{-1}K(H^{-1}x)$ is the scaled kernel, and $H$ is a positive-definite bandwidth matrix.

Facts

Widely used kernels