Policy

Definition

Deterministic Policy

$\pi :S\to A,\pi (s)$ A deterministic policy $\pi$ is a mapping from state space $S$ to action space $A$.

Stochastic Policy

$\pi (a|s)=P(A_t=a|S_t=s)$ A stochastic policy $\pi$ is a probability distribution over actions $A$ for a given state $S$.

Optimal Policy

${\pi }_{\ast }(s)=\underset{a}{\mathrm{argmax}}{q}_{\ast }(s,a)$ A policy maximizing the cumulative sum of rewards is called an optimal policy.

Better Policy

Consider two policies $\pi$ and ${\pi }^{\prime }$. ${\pi }^{\prime }$ is better policy than $\pi$ if $\forall s,v_{{\pi }^{\prime }}(s)\ge v_{\pi }(s)$

Facts

• Under known MDP, there exists deterministic optimal policy ${\pi }_{\ast }(s)$.
• Under unknown MDP, stochastic policy is needed.