CycleGAN

Definition

Cycle GAN is unsupervised image-to-image translation model. The model can be trained without paired training examples.

Architecture

CycleGAN consists of two GANs $G_{x\to y}:X\to Y$ and $G_{y\to x}:Y\to X$, and is trained by minimizing the cycle consistency loss for each image from both domain $X$ and $Y$ $G_{y\to x}(G_{x\to y}(x))\approx x\text{and}{G}_{x\to y}(G_{y\to x}(y))\approx y$

Objective Function

The objective function of CycleGAN consists of three losses: adversarial loss ($x\to y$), adversarial loss ($y\to y$), and cycle-consistency loss.

Adversarial loss ($x\to y$): $L_{\text{GAN}}(G_{x\to y},D_y,X,Y)=E_{y\sim p_y(y)}[\ln D_y(y)]+E_{x\sim p_x(x)}[\ln (1-D_y(G_{x\to y}(x)))]$ Adversarial loss ($y\to x$): $L_{\text{GAN}}(G_{y\to x},D_x,Y,X)=E_{x\sim p_x(x)}[\ln D_x(x)]+E_{y\sim p_y(y)}[\ln (1-D_x(G_{y\to x}(y)))]$ Cycle-consistency loss: $L_{\text{cycle}}(G_{x\to y},G_{Y\to x})=E_{x\sim p_x(x)}[||x-G_{y\to x}(G_{x\to y}(x))|{|}_1]+E_{y\sim p_y(y)}[||y-G_{x\to y}(G_{y\to x}(y))|{|}_1]$ where:

• $G$ is the generator
• $D$ is the discriminator
• $p_x(x)$ is the distribution of the data from domain $X$
• $p_y(y)$ is the distribution of the data from domain $Y$

The full objective function for CycleGAN can be written as:

${\min }_{{\theta }_{G_{x\to y}},{\theta }_{G_{y\to x}}}{\max }_{{\theta }_{D_x},{\theta }_{D_y}}[\underset{\text{Adversarial Loss}(x\to y)}{\underbrace{L_{\text{GAN}}(G_{x\to y},D_y,X,Y)}}+\underset{\text{Adversarial Loss}(y\to x)}{\underbrace{L_{\text{GAN}}(G_{y\to x},D_x,Y,X)}}+\lambda \underset{\text{Cycle-Consistency Loss}}{\underbrace{L_{\text{cycle}}(G_{x\to y},G_{Y\to x})}}]$ where $\lambda$ is the weight for the cycle consistency loss.