Segmenter

Definition

Segmenter is a Transformer-based semantic segmentation model.

Architecture

The encoder of Segmenter has a ViT architecture.

The decoder takes the output of the encoder and the class tokens as inputs. The embedded token matrix ($N\times D$) and the class mask matrix ($K\times D$) are multiplied and Softmax Function is applied on the class dimension to obtain pixel-wise class scores, where $N$ is the number of patches, $D$ is the channel dimension, $K$ is the number of classes.