PPO's clipped surrogate objective (Schulman et al. 2017) over a batched rollout. More...

#include <cstdint>
#include <vector>
#include "pulsatrix/device_backend.hpp"
#include "pulsatrix/tensor.hpp"

Include dependency graph for ppo_clipped_loss.hpp:

Classes
class	pulsatrix::PPOClippedLoss
	loss = mean_b( -min( r_b * A_b, clamp(r_b, 1-eps, 1+eps) * A_b ) ), where `r_b = pi_new(a_b\|s_b) / pi_old(a_b\|s_b)` – the clipped surrogate objective of Schulman et al. 2017 (arXiv:1707.06347), negated so that minimizing it maximizes the objective the paper states. More...

Detailed Description

PPO's clipped surrogate objective (Schulman et al. 2017) over a batched rollout.

Classes