Relational GCN

Definition

The Relational graph convolutional network (RGCN) is an extension of the GCN designed to handle the graph with multiple types of edges or relationships between nodes.

Architecture

The model uses relation-specific wrights matrices. The information of neighbors of a node is aggregated considering the different relation types.

$h_i^{(l+1)}=\sigma \left(\sum _{r\in R}\sum _{v_j\in {\mathcal{N}}_r(v_i)}\left[\frac{1}{||{\mathcal{N}}_r(v_i)|{|}_2}{W}_r^{(l)}{h}_j^{(l)}\right]+W_0^{(l)}{h}_i^{(l)}\right)$ where:

• $h_i^{(l)}$ is the feature vector of node $v_i$ at layer $l$
• $R$ is the set of relations (edge types) in the graph
• ${\mathcal{N}}_r(v_i)$ is the neighbors of the node $v_i$ under relation $r$.
• $W_r^{(l)}$ is the learnable weight matrix for relation $r$ at layer $l$ ($W_0^{(l)}$ is for self-connection).
• $\sigma$ is a non-linear Activation Function

Regularization Methods

The number of parameters of the model is increased rapidly with respect to the number of relations. I cause overfitting and scability issues. The techniques such as basis decomposition or block-diagonal decomposition of the weight matrices can be employed to reduce the number of parameters and improve model efficiency.

Basis Decomposition

In basis decomposition, instead of learning a separate weight matrix for each relation, we express each relation-specific weight matrix as a linear combination of a smaller set of basis matrices.

$W_r={\sum }_{b=1}^B{a}_{rb}{V}_b$ where:

• $B$ is the number of basis matrices (typically $B\ll |R|$)
• $V_b$ are the learnable basis matrices
• $a_{rb}$ are learnable importance weight of matrix $V_b$.

This method reduces the number of parameters from $O(|R|{d^{(l)}}^2)$ to $O(B{d^{(l)}}^2+|R|B)$, where $d^{(l)}$ is the hidden dimension in layer $l$.

Block-Diagonal Decomposition

In block-diagonal decomposition, each relation-specific weight matrix is constrained to be block-diagonal

$W_r=\text{diag}(Q_{r1},\dots ,Q_{rB})=\left[\begin{array}{ccc}{Q}_{r1}& & O\\ & \ddots & \\ O& & Q_{rB}\end{array}\right]$ where each $Q_{rb}$ is a low-dimensional matrix of size $(d/B)\times (d/B)$.

This method reduces the number of parameters from $O(|R|{d^{(l)}}^2)$ to $O(|\mathcal{R}|\frac{{d^{(l)}}^2}{B})$, where $d^{(l)}$ is the hidden dimension in layer $l$.

The channels (dimensions) of the node feature vector are indeed grouped The transformation of the channel values occurs within each group

It can be viewed as grouping the channels of the node feature vector and transforming the channel values within each group.