Needs help on understanding `buffer_size` and `train_batch_size`

[mannyv]

@albheim,

Perhaps this post will be useful:

PPO algorithms train buffer only collects the first fragment from each worker?

@mickelliu, This has a pretty good overview of how sample collection works: https://docs.ray.io/en/latest/rllib-sample-collection.html During the execution plan each worker will be asked to produce samples from new rollouts of the current model. Assuming you are using the truncate_episodes batch_mode, each worker will roll out exactly num_envs_per_worker * rollout_fragment_length that is how you get a total sample size of num_workers*num_envs_per_worker * rollout_fragment_length. RLLIB requi…