Fot
June 15, 2021, 10:44pm
1
Hi all! I am a bit confused about the role of the timesteps_per_iteration parameter, and how is it related to the train_batch_size parameter. Could someone please clarify their differences?
2 Likes
Hi,
I also wanted top knownwhat it is exactly doing.
opened 04:13AM - 29 Mar 19 UTC
closed 12:33AM - 02 Apr 19 UTC
question
Hi,
I wanted to know more about what the `timesteps_per_iteration` parameter… means in context of DQN based agents? Quick glance at the code: https://github.com/ray-project/ray/blob/cff08e19ff1606ef6e718624703e8e0da19b223d/python/ray/rllib/agents/dqn/dqn.py#L257-L261
suggests that you optimize for `timesteps_per_iteration` after each env step? That doesn't quite seem right since the default value for `timesteps_per_iteration` is 1000 which seems high .
Also, does `timesteps_per_iteration` have a different connotation in case of distributed agents like ApeX?
1 Like