Convolutional LSTM

Definition

The input and hidden state of convolutional LSTM have a matrix form, and the fully connected layer in the LSTM cell is substituted with convolutional layer.

• $f_t=\sigma (W_{xf}\ast {\mathcal{X}}_t+W_{hf}\ast {\mathcal{H}}_{t-1}+W_{cf}\odot {\mathcal{C}}_{t-1}+b_f)$
• $i_t=\sigma (W_{xi}\ast {\mathcal{X}}_i+W_{hi}\ast {\mathcal{H}}_{t-1}+W_{ci}\odot {\mathcal{C}}_{t-1}+b_f)$
• $o_t=\sigma (W_{xo}\ast {\mathcal{X}}_t+W_{ho}\ast {\mathcal{H}}_{t-1}+W_{co}\odot {\mathcal{C}}_{t-1}+b_o)$
• ${\mathcal{C}}_t=f_t\odot {\mathcal{C}}_{t-1}+i_t\odot \tanh (W_{xc}\ast {\mathcal{X}}_t+W_{hc}\ast {\mathcal{H}}_{t-1}+b_c)$
• ${\mathcal{H}}_t=o_t\odot \tanh ({\mathcal{C}}_t)$ where $W_{..}$ are convolution filters.