Distributional Bellman Equation

Definition

Distributional Bellman Equation

$Z^{\pi }(x,a)\stackrel{D}{=}R(x,a)+\gamma Z^{\pi }(X^{\prime },A^{\prime })$

Distributional Bellman Operator

$({\mathcal{T}}^{\pi }Z)(x,a)\stackrel{D}{:=}R(x,a)+\gamma Z(X^{\prime },A^{\prime })$

The distributional Bellman equation can be written with the operator as $Z^{\pi }={\mathcal{T}}^{\pi }{Z}^{\pi }$