When using RLlib with configuration parameters unchanged, I see in the reports that timesteps_total=4000 per iteration when using ‘PPO’ or ‘A2C’ and timesteps_total=10000 per iteration when using ‘TD3’.
Is there a way to programatically get this timesteps_total value before a “run”? (I mean the expected timesteps_total value, as of course, the real value will be obtained when the “run” is done.)
For PPO, the number of time steps per iteration depend on config["train_batch_size"], which defaults to 4000: RLlib Algorithms — Ray v1.4.0
So you can use the config dict to programmatically change or read the time steps per iteration.
For A2C and TD3, I’m not so sure how the time steps are determined. Here are the default config values:
https://docs.ray.io/en/latest/rllib-algorithms.html#advantage-actor-critic-a2c-a3c
https://docs.ray.io/en/latest/rllib-algorithms.html#deep-deterministic-policy-gradients-ddpg-td3