SENet

Definition

SENet is a deep Convolutional Neural Network architecture that explicitly modeling interdependencies between channels.

Architecture

SE Block

SE block consists of the two operations: squeeze and excitation operations. The original feature maps are rescaled using the channel-wise weights produced by the excitation operation. SE block can be integrated into various existing architectures, not just used as a standalone network.

Squeeze Operation

The squeeze operation aggregates feature maps across spatial dimensions to produce a channel descriptor. It’s typically done using Global Average Pooling. For a given feature map of size $h\times w\times c$, the squeeze operation produces a $1\times 1\times c$ vector.

Excitation Operation

The excitation operation takes the output of the squeeze operation and produces a set of per-channel modulation weights. It’s typically implemented using two fully connected layers with a non-linearity in between. $s=\sigma (W_2\mathrm{ReLU}(W_{1}z))$ where $\sigma$ is a Sigmoid Function, and $W_1$ and $W_2$ are learnable parameters.