normalized advantage function

References:

• Gu et al., Continuous Deep Q-Learning with Model-based Acceleration (2016).

Instead of modeling $Q(s, a)$ directly, we build a network that outputs $V(s)$ and quantities $\mu(s), P(s)$ defining a quadratic representation of the advantage

which together imply $Q(s, a) = V(s) + A(s, a)$. Using a quadratic to represent the advantage allows us to immediately read off the argmax action $\mu(s)$. Effectively, $\mu(s)$ is a policy network,