InfoGAN

Definition

Information maximizing GAN (InfoGAN) is an extension of the GAN architecture that aims to learn disentangled representations in an unsupervised manner. InfoGAN introduces a latent code $c$ in addition to the noise vector $z$. This latent code is designed to capture interpretable and meaningful features of the generated data.

Architecture

The core idea of InfoGAN is to maximize the Mutual Information between the latent code $c$ and the generated samples $G(z,c)$. This encourages the model to learn representations $c$ corresponding to semantic features of the generated output. InfoGAN introduces an auxiliary network $Q$ that attempts to recover the latent code $c$ from the generated samples.

Objective Function

GAN loss: $L_{GAN}=E_{x\sim p_r(x)}[\ln D(x)]+E_{z\sim p_n(z),c\sim p_n(c)}[\ln (1-D(G(z,c)))]$ Mutual Information $I(c;G(z,c))$ and variational lower bound $V(G,Q)$: $I(c;G(z,c))\ge V(G,Q)=E_{z\sim p_n(z),c\sim p_n(c)}[\ln Q(c|G(z,c))]+H(c)$ where:

• $G$ is the generator
• $D$ is the discriminator
• $Q$ is the auxiliary network
• $p_r$ is the distribution of the input data
• $p_n$ is the distribution of noise
• $H(c)$ is the Entropy of the latent code distribution $p_n(c)$

The full objective function for InfoGAN is defined as: ${\min }_{{\theta }_G{\theta }_Q}{\max }_{{\theta }_D}[L_{GAN}(D,G)-\lambda V(G,Q)]$ Where:

• $L_{GAN}$ is the standard GAN objective
• $V$ is the approximation of the mutual information between $c$ and $G(z,c)$
• $\lambda$ is a hyperparameter controlling the importance of the mutual information term