Fully-Connected LSTM

Definition

Fully-connected LSTM introduced additional connections from the cell state to the FC layers for each gate in LSTM cell.

Architecture

where:

• $f_t=\sigma (W_f{x}_t+U_f{h}_{t-1}+V_f{c}_{t-1}+b_f)$
• $i_t=\sigma (W_i{x}_t+U_i{h}_{t-1}+V_i{c}_{t-1}+b_i)$
• $o_t=\sigma (W_o{x}_t+U_o{h}_{t-1}+V_o{c}_{t-1}+b_o)$
• $c_t=f_t\odot c_{t-1}+i_t\odot \tanh (W_c{x}_t+U_c{h}_{t-1}+b_c)$
• $h_t=o_t\odot \tanh (c_t)$