# "Iteration always 1" challenge

**URL:** <https://discuss.ray.io/t/iteration-always-1-challenge/10347>\
**Category:** Ray Tune\
**Created:** [April 22, 2023, 12:32am UTC](https://discuss.ray.io/t/iteration-always-1-challenge/10347 "2023-04-22T00:32:25Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![PhilippWillms](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/philippwillms/32/3885_2.png) [@PhilippWillms](https://discuss.ray.io/u/PhilippWillms)\
**Post date:** [April 22, 2023, 12:32am UTC](https://discuss.ray.io/t/iteration-always-1-challenge/10347/1 "2023-04-22T00:32:25Z")

</div>

Even though I read the Tune FAQ article [Why are all my trials returning “1” iteration?](https://docs.ray.io/en/latest/tune/faq.html#why-are-all-my-trials-returning-1-iteration), I am concerned that my implementation is working. Indeed I face the topic of all 5 trials returning 1 iteration.

How can I control that algorithm training has been properly executed? Should I look at the `num_sgd_iter`?

```auto
    tuner = tune.Tuner(
        "PPO",
        param_space=config,
        run_config=RunConfig(
            stop=stopping_criteria,
            checkpoint_config=CheckpointConfig(
                checkpoint_score_attribute="episode_reward_mean",
                checkpoint_score_order="max",
                checkpoint_frequency=2,
            ),
        ),
        tune_config=tune.TuneConfig(
            metric="episode_reward_mean",
            mode="max",
            num_samples=5,
            reuse_actors=False,
            max_concurrent_trials=3
        ),
```

---

<div class="post-metadata">

**Author:** ![kai](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/kai/32/3380_2.png) [@kai](https://discuss.ray.io/u/kai)\
**Post date:** [April 24, 2023, 10:53am UTC](https://discuss.ray.io/t/iteration-always-1-challenge/10347/2 "2023-04-24T10:53:30Z")

</div>

The interesting part here would be the defined `stopping_criteria`. Can you share those with us?

RLlib returns a number of metrics that can be checked to see if training took place, but yes, the `num_sgd_iter` is a good candidate to see how often SGD optimization has been triggered.

---

<div class="post-metadata">

**Author:** ![PhilippWillms](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/philippwillms/32/3885_2.png) [@PhilippWillms](https://discuss.ray.io/u/PhilippWillms)\
**Post date:** [April 24, 2023, 9:18pm UTC](https://discuss.ray.io/t/iteration-always-1-challenge/10347/3 "2023-04-24T21:18:59Z")

</div>

The stopping criteria are as follows, to be finetuned:

```auto
stopping_criteria = {
        "training_iteration": 5,
        "timesteps_total": 6,
        "episode_reward_mean": 5,
    }

```

While the `iter` in the trial summary output always remains 1, indeed the `num_sgd_iter` and the ts are different. However, the `num_sgd_iter` is a tuning parameter initialized by

```auto
"num_sgd_iter": tune.randint(100, 1000),

```

---

<div class="post-metadata">

**Author:** ![kai](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/kai/32/3380_2.png) [@kai](https://discuss.ray.io/u/kai)\
**Post date:** [April 25, 2023, 9:36am UTC](https://discuss.ray.io/t/iteration-always-1-challenge/10347/4 "2023-04-25T09:36:17Z")

</div>

The stopping criteria are OR conditions, i.e. as soon as one of the conditions applies, the training is stopped. I would assume that `timesteps_total` achieves `6` in the first iteration, hence you see only one result.

If you comment out timesteps\_total and episode\_reward\_mean from the stopping criteria, each trial should run to 5 iterations.