Denoising Autoencoder

Definition

$D(x^{\prime })=g(f(x^{\prime }))=\approx x$ where $x^{\prime }=x+ϵ$, here $ϵ$ is a random noise, $f$ is an encoder, and $g$ is a decoder.

Denoising autoencoder is a type of Autoencoder designed to learn robust representations of data by reconstructing clean inputs from noisy versions.

Architecture

The model takes clean input $x$ and adds noise to create corrupted input $x^{\prime }$. The noisy input is fed into the mdoel, and the model attempts to reconstruct the original clean input by minimizing MSE. $L(x,g(f(x^{\prime })))=||x-g(f(x^{\prime }))|{|}^2$

In a Gaussian noise setting, the estimator minimizing MSE is the mean of posterior distribution $E[x|x^{\prime }]$, so DAE learns a function that approximates the posterior mean $D(x^{\prime })\approx E[x|x^{\prime }]$. Therefore, using a DAE with Tweedie’s Formula, we can estimate the Score Function of a sample space.