Convolutional GRU

Definition

Convolutional GRU applied the idea of Convolutional LSTM to Gated Recurrent Unit. When stacking multiple ConvGRU layers, each layer takes previous layer’s hidden state as an extra input.

• $r_t=\sigma (W_{xr}\ast x_t+W_{h{r}^l}^l\ast h_t^{l-1}+W_{hr}\ast h_{t-1})$
• $z_t^l=\sigma (W_{xz}^l\ast x_t+W_{h{z}^l}^l\ast h_t^{l-1}+W_{hz}^l\ast h_{t-1})$
• $h_t=(1-z_t)\odot h_{t-1}+z_t\odot \tanh (W_{hx}\ast x_t+W_{hh}\ast (r_t\odot h_{t-1}))$ where $W_{..}$ are convolution filters.