Distributional Bellman Optimality Equation

Definition

Distributional Bellman Optimality Equation

$Z^{\ast }(x,a)\stackrel{D}{=}R(x,a)+\gamma Z^{\ast }(X^{\prime },\underset{a^{\prime }}{\mathrm{argmax}}E[Z^{\ast }(X^{\prime },a^{\prime })])$

Distributional Bellman Optimality Operator

$\mathcal{T}Z(x,a)\stackrel{D}{:=}R(x,a)+\gamma Z(X^{\prime },\underset{a^{\prime }}{\mathrm{argmax}}E[Z(X^{\prime },a^{\prime })])$

The distributional Bellman optimality equation can be written with the operator as $Z^{\ast }={\mathcal{T}}^{\pi }{Z}^{\ast }$