# Limit number of steps? **URL:** \ **Category:** Ray Tune\ **Created:** [May 3, 2022, 1:41am UTC](https://discuss.ray.io/t/limit-number-of-steps/6003 "2022-05-03T01:41:00Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Christian\_Coletti](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/christian_coletti/32/2472_2.png) [@Christian\_Coletti](https://discuss.ray.io/u/Christian_Coletti)\ **Post date:** [May 3, 2022, 1:41am UTC](https://discuss.ray.io/t/limit-number-of-steps/6003/1 "2022-05-03T01:41:00Z")
**How severe does this issue affect your experience of using Ray?** - Medium: It contributes to significant difficulty to complete my task, but I can work around it. Ray’s `tune.run()` is running endlessly. Can I force it to save a checkpoint and begin a new iteration after a number of steps? ```auto analysis = tune.run( "PPO", stop={ "episode_reward_mean": 2, "training_iteration": 35, }, config={ "env": "TradingEnv", "env_config": env_config_training, "log_level": "ERROR", #"log_level": "INFO", #"log_level": "DEBUG", "framework": "torch", "ignore_worker_failures": False, "clip_rewards": True, "lr": LR, "lr_schedule": [ [0, 1e-1], [int(1e2), 1e-2], [int(1e3), 1e-3], [int(1e4), 1e-4], [int(1e5), 1e-5], [int(1e6), 1e-6], [int(1e7), 1e-7] ], "model": { "use_lstm": True, "lstm_cell_size": 512 }, "gamma": GAMMA, "observation_filter": "MeanStdFilter", "lambda": LAMBDA, "vf_share_layers": True, "vf_loss_coeff": VF_LOSS_COEFF, "entropy_coeff": ENTROPY_COEFF, "evaluation_interval": 1, # Run evaluation on every iteration "evaluation_config": { "env_config": env_config_evaluation, # The dictionary we built before (only the overriding keys to use in evaluation) "explore": False, # We don't want to explore during evaluation. All actions have to be repeatable. }, }, metric=checkpoint_metric, mode="max", search_alg=search_alg, scheduler=scheduler, num_samples=10, # Samples per hyperparameter combination. More averages out randomness. Less runs faster keep_checkpoints_num=10, # Keep the last 10 checkpoints checkpoint_freq=1, # Do a checkpoint on each iteration (slower but you can pick more finely the checkpoint to use later) # resume="AUTO", local_dir="./results", name=f"testing_{int(time.time()-1651400000)}", trial_name_creator=Methods.trial_name_string ) ``` ---
**Author:** ![xwjiang2010](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/xwjiang2010/32/1476_2.png) [@xwjiang2010](https://discuss.ray.io/u/xwjiang2010)\ **Post date:** [May 3, 2022, 4:06am UTC](https://discuss.ray.io/t/limit-number-of-steps/6003/2 "2022-05-03T04:06:29Z")
> [@Christian\_Coletti](#): > > `training_iteration` Does `timesteps_total` under `stop` work for you? ---
**Author:** ![Christian\_Coletti](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/christian_coletti/32/2472_2.png) [@Christian\_Coletti](https://discuss.ray.io/u/Christian_Coletti)\ **Post date:** [May 5, 2022, 1:19am UTC](https://discuss.ray.io/t/limit-number-of-steps/6003/3 "2022-05-05T01:19:16Z")
I will try it. I’m surprised the option is not listed in documentation. [https://docs.ray.io/en/latest/tune/api\_docs/stoppers.html](https://docs.ray.io/en/latest/tune/api_docs/stoppers.html) [https://docs.ray.io/en/latest/tune/tutorials/tune-stopping.html](https://docs.ray.io/en/latest/tune/tutorials/tune-stopping.html) ---
**Author:** ![Christian\_Coletti](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/christian_coletti/32/2472_2.png) [@Christian\_Coletti](https://discuss.ray.io/u/Christian_Coletti)\ **Post date:** [May 5, 2022, 1:38am UTC](https://discuss.ray.io/t/limit-number-of-steps/6003/4 "2022-05-05T01:38:05Z")
> [@xwjiang2010](#): > > > [@Christian\_Coletti](#): > > > > `training_iteration` > > Does `timesteps_total` under `stop` work for you? It does not, for some reason tune.run() ignores it and continues training on multiple timesteps even when `"timesteps_total": 1` is under `stop`. ```auto analysis = tune.run("PPO", stop={ "timesteps_total": 1, }, ...) # does not work ``` ---
**Author:** ![xwjiang2010](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/xwjiang2010/32/1476_2.png) [@xwjiang2010](https://discuss.ray.io/u/xwjiang2010)\ **Post date:** [May 5, 2022, 5:14pm UTC](https://discuss.ray.io/t/limit-number-of-steps/6003/5 "2022-05-05T17:14:52Z")
@arturn When you get a chance, could you hep @Christian_Coletti with this question? ---
**Author:** ![arturn](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/arturn/32/2096_2.png) [@arturn](https://discuss.ray.io/u/arturn)\ **Post date:** [May 6, 2022, 11:20am UTC](https://discuss.ray.io/t/limit-number-of-steps/6003/6 "2022-05-06T11:20:48Z")
Hey @Christian_Coletti, hey @xwjiang2010 , The option is not listed under the tune api docs, because it’s specific to RLLib. So `timesteps_total` is not a generic metric for everything tunable by tune. Every iteration, you can have a look at the output of the analysis and will find something like this … ```auto ... timestamp: 1651834198 timesteps_since_restore: 0 timesteps_total: 12000 training_iteration: 3 trial_id: e577a_00000 ... ``` … amongst many other metrics and infos. These are the ones that you can chose for the given algorithm. They vary per algorithm, but training\_iteration or timesteps\_total are obviously ubiquitous. Regarding your experiment: To reproduce I had to cut out a couple of config parameters and have used the following: ```auto from ray import tune analysis = tune.run( "PPO", stop={ "timesteps_total": 10, "episode_reward_mean": 45, "training_iteration": 20, }, config={ "env": "CartPole-v0", "log_level": "ERROR", "framework": "torch", "ignore_worker_failures": False, "clip_rewards": True, "lr_schedule": [ [0, 1e-1], [int(1e2), 1e-2], [int(1e3), 1e-3], [int(1e4), 1e-4], [int(1e5), 1e-5], [int(1e6), 1e-6], [int(1e7), 1e-7] ], "model": { "use_lstm": True, "lstm_cell_size": 512 }, "observation_filter": "MeanStdFilter", "evaluation_interval": 1, }, mode="max", keep_checkpoints_num=10, checkpoint_freq=1, local_dir="~/ray_results/", name="test", resume=False, ) ``` This works on my side. The experiment stops after the first iteration, because the timesteps are exceeded. Similar logic works for `episode_reward_mean` or `training_iteration`. Can you confirm or provide a script that does not provide the desired behaviour out of the box? Ideally with the version of ray your are using 🙂 Cheers ---
**Author:** ![Christian\_Coletti](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/christian_coletti/32/2472_2.png) [@Christian\_Coletti](https://discuss.ray.io/u/Christian_Coletti)\ **Post date:** [May 6, 2022, 3:15pm UTC](https://discuss.ray.io/t/limit-number-of-steps/6003/7 "2022-05-06T15:15:06Z")
Hello, My `tune.run()` is running endlessly and not beginning new iterations. I’m looking for a way to shorten an iteration and force it to save a checkpoint. Since you said > [@arturn](#): > > The experiment stops after the first iteration, because the timesteps are exceeded. I believe this does not answer my question Thanks ---
**Author:** ![arturn](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/arturn/32/2096_2.png) [@arturn](https://discuss.ray.io/u/arturn)\ **Post date:** [May 7, 2022, 10:31pm UTC](https://discuss.ray.io/t/limit-number-of-steps/6003/8 "2022-05-07T22:31:26Z")
No problem! Is my script running endlessly on your side? Can you confirm or provide a script that does not provide the desired behaviour out of the box? Ideally with the version of ray your are using 🙂 Your script is not executable as is and removing the unknowns to run somethings similar does not yield your undesired outcome. Is there maybe a misunderstanding? I understand that tune never ends the first training iteration and is therefore never able to checkpoint - since there is not result. Correct? To shorten iterations you will have to go through RLlib which has traditionally used the `"timesteps_per_iteration"` and has now switched to `"min_sample_timesteps_per_reporting"` and `"min_train_timesteps_per_reporting"` incase you are working with a nightly build. Best