Is there a way to set num_env_steps_sampled?

RLlib
1
  • Low: It annoys or frustrates me for a moment.

I’m now trying to train a PPO agent in custom env, the step() will cost about 3min.

algo = PPOConfig()
algo = algo.environment(env =CustomEnv)
algo = algo.framework('torch').build().train()

I notice that in result of train there has num_env_steps_sampled

...
num_agent_steps_sampled: 4000
num_agent_steps_trained: 4000
num_env_steps_sampled: 4000
num_env_steps_trained: 4000
num_env_steps_sampled_this_iter: 4000
num_env_steps_trained_this_iter: 4000
timesteps_total: 4000
num_steps_trained_this_iter: 4000
...

Is there a way to set num_env_steps_sampled?(for PPO and other built-in algorithms)
What’s the best practice of training in expensive step() env?