Generative Adversarial Networks

Definition

Generative adversarial network (GAN) is a type of Neural Network model generating data that resembles training data.

Architecture

Training

In the training process of GAN, the two neural networks compete against each other. The generator (G) generates fake data and the discriminator (D) tries to distinguish between real and fake data.

1. The generator creates fake data from random noise.
2. The discriminator is shown both real data from the training set and fake data from the generator. It tries to classify which is which.
3. Based on how well the discriminator performs, both networks receive feedback
   • The generator aims to improve its fake data to fool the discriminator.
   • The discriminator aims to get better at distinguishing real from fake.
4. This process continues iteratively, with both networks improving over time.

Objective Function

The objective function of GAN is defined as ${\min }_{{\theta }_G}\underset{\approx D_{JS}(p_r,p_z)}{\underbrace{\underset{{\theta }_D}{\max }[E_{x\sim p_r(x)}[\ln D(x)]+E_{z\sim p(z)}[\ln (1-D(G(z)))]]}}$ where:

• $G$ is the generator
• $D$ is the discriminator
• $p_r$ is the distribution of the input data
• $p_z$ is the distribution of noise

Facts

When the discriminator is optimal, minimizing the objective function with respect to the generator is equivalent to minimizing the JS-Divergence between the real data distribution $p_r(x)$ and the generated distribution $G(z),z\sim p_z(z)$.