n-Step TD

Definition

$V(S_t)\leftarrow V(S_t)+\alpha [G_{t:t+n}-V(S_t)]$ where $G_{t:t+n}$ is the n-Step Return.

The n-step TD learning is a method that bridges the gap between TD and MC methods.

Examples

• $n=1$: Temporal Difference Learning
• $n=\infty$: Monte Carlo Method

Larger $n$ reduces bias but increases variance.