Hello everyone, as the title suggests, I’m trying to understand how these two parameters work for any off-policy algorithms such as QMIX. I have read a few posts and the doc but I still have difficulties fully understand the usage. From my experiences of running QMIX on my custom Gym environment ye…

Hi mickelliu! On train_batch_size, from the documentation : The size of that train batch is determined by the train_batch_size config parameter. Train batches are usually sent to the Policy’s learn_on_batch method, which handles loss- and gradient calculations, and optimizer stepping. The trainin…

Thanks @arturn for your kind response. I have another question if you don’t mind. My current understanding is that the concat_train_batch, which is a concatenation of all batches, it gets passed down to the SGD optimizer and updates the network once. So if train_batch_size is a hard limit on the si…

I’m also wondering about some of these settings. So if we have rollout_fragment_length and num_workers and set train_batch_size = num_workers * rollout_fragment_length it should work nicely and make one large batch for training. But if we set train_batch_size to something smaller than this, will it…

@albheim , Perhaps this post will be useful: [image] PPO algorithms train buffer only collects the first fragment from each worker? RLlib @mickelliu , This has a pretty good overview of how sample collection works: https://docs.ray.io/en/latest/rllib-sample-collection.ht…

Needs help on understanding `buffer_size` and `train_batch_size`

RLlib

mannyv October 30, 2021, 11:47am 5

@albheim,

Perhaps this post will be useful:

Topic		Replies	Views
PPO algorithms train buffer only collects the first fragment from each worker? RLlib	4	821	October 30, 2021
[RLlib] Batch size for complete_episodes issue RLlib	6	2316	February 3, 2022
PPO configuration parameters: num_rollout_workers & train_batch_size Configure Algorithm, Training, Evaluation, Scaling	1	847	November 2, 2023
Does the agent train per episode or per iteration RLlib	1	586	November 1, 2021
RLLib PPO Trainer allocating additional memory on second training iteration RLlib	0	309	July 21, 2022

Needs help on understanding `buffer_size` and `train_batch_size`

Related topics