# Understanding the Stopping Process for ray.rllib.agents.dqn.DQNTrainer.train()

**URL:** <https://discuss.ray.io/t/understanding-the-stopping-process-for-ray-rllib-agents-dqn-dqntrainer-train/2203>\
**Category:** RLlib\
**Created:** [May 19, 2021, 3:00pm UTC](https://discuss.ray.io/t/understanding-the-stopping-process-for-ray-rllib-agents-dqn-dqntrainer-train/2203 "2021-05-19T15:00:28Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Saurabh\_Arora](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/saurabh_arora/32/1012_2.png) [@Saurabh\_Arora](https://discuss.ray.io/u/Saurabh_Arora)\
**Post date:** [May 19, 2021, 3:00pm UTC](https://discuss.ray.io/t/understanding-the-stopping-process-for-ray-rllib-agents-dqn-dqntrainer-train/2203/1 "2021-05-19T15:00:28Z")

</div>

Hi everyone.

I am using default config setting for ray.rllib.agents.dqn.DQNTrainer(config=config, env=select\_env)  
When I run one execution of DQNTrainer.train() method, it gives an arbitrary number of episodes and arbitrary number of training iterations.

Can someone please help me understand step-by-step how does DQNTrainer.train() method decide to stop?

---

<div class="post-metadata">

**Author:** ![kai](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/kai/32/3380_2.png) [@kai](https://discuss.ray.io/u/kai)\
**Post date:** [May 19, 2021, 3:05pm UTC](https://discuss.ray.io/t/understanding-the-stopping-process-for-ray-rllib-agents-dqn-dqntrainer-train/2203/2 "2021-05-19T15:05:08Z")

</div>

cc @sven1977 can you help with this?

---

<div class="post-metadata">

**Author:** ![Saurabh\_Arora](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/saurabh_arora/32/1012_2.png) [@Saurabh\_Arora](https://discuss.ray.io/u/Saurabh_Arora)\
**Post date:** [May 19, 2021, 8:47pm UTC](https://discuss.ray.io/t/understanding-the-stopping-process-for-ray-rllib-agents-dqn-dqntrainer-train/2203/3 "2021-05-19T20:47:21Z")

</div>

@sven1977 , I will really appreciate any help on this.

---

<div class="post-metadata">

**Author:** ![sven1977](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/sven1977/32/53_2.png) [@sven1977](https://discuss.ray.io/u/sven1977)\
**Post date:** [May 24, 2021, 1:47pm UTC](https://discuss.ray.io/t/understanding-the-stopping-process-for-ray-rllib-agents-dqn-dqntrainer-train/2203/4 "2021-05-24T13:47:44Z")

</div>

Hey @Saurabh_Arora , thanks for this question!

You always get one training iteration upon calling `trainer.train()`. You can check this via `trainer.iteration` afterwards.

The number of episodes depends on the environment (does it stop early?, etc…).

And env steps depend on the algorithm’s execution plan. Some algos perform a fixed set of update steps per iteration, e.g. PPO. Others have a timing mechanism, like DQN, IMPALA.

In particular, DQN has this here in its plan (`ray/rllib/agents/dqn/dqn.py`):

```auto
    train_op = Concurrently(
        [store_op, replay_op],
        mode="round_robin",
        output_indexes=[1],
        round_robin_weights=calculate_rr_weights(config)) # <- these weights here determine the ratio between a) sample-collection-and-storing and b) sampling-from-buffer-and-train-one-step updates

```

The round robin weighting causes each iteration to contain a slightly different number of steps.

---

<div class="post-metadata">

**Author:** ![Saurabh\_Arora](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/saurabh_arora/32/1012_2.png) [@Saurabh\_Arora](https://discuss.ray.io/u/Saurabh_Arora)\
**Post date:** [May 26, 2021, 1:47pm UTC](https://discuss.ray.io/t/understanding-the-stopping-process-for-ray-rllib-agents-dqn-dqntrainer-train/2203/5 "2021-05-26T13:47:07Z")

</div>

@sven1977 , thanks for responding. Can I customize stopping criterion of `trainer.train()` while running PPO (or DQN)?