rl diagnostics

Things that might be useful to log in a reinforcement learning algorithm:

1. Return of each trajectory. (summarize as mean/std/min/max)
2. Value estimate of the initial state (should converge to the average return)
3. Trajectory lengths.
4. temporal difference error at each step (summarize by mean TD error, and std/min/max. mean should converge to zero)
5. for policy gradient methods:
   1. action space entropy
   2. KL divergence in policy from one epoch to the next.