How to interactively stop running trials

Ray Libraries (Data, Train, Tune, Serve) Ray Tune
1

How to interactively stop running trials when using tune.run()? Stopping certain ones at my choosing. I usually monitor the live training via tensorboard. I start a RLlib training similar to below. There are custom_metrics captured via callbacks. I know about the stop config dict for tune.run(). However, sometimes I just want to interactively stop some trials because I am actively experimenting.

import ray
from ray import tune

from ray.rllib.examples.env.random_env import RandomEnv

config = {
  "env": RandomEnv,
  "lr" : tune.grid_search([1e-3, 1e-4, 1e-5, 1e-6]),
}

stop = {
  "training_iteration": 5,
}

ray.init()


results = tune.run(
  "A3C",
  name="test-lr_schedule",
  config=config, 
  stop=stop, 
)

ray.shutdown()
2

You could use the Tune Client API: Tune Client API — Ray v2.0.0.dev0

1 Like
3

Hello together,

I want to reopen this topic, since I’m trying to figure out how to use the Tune Client API with a Tuner object. As described here https://docs.ray.io/en/latest/tune/api/client.html it is necessary to pass a server_port to the tune.run() call. But how is it supposed to do with a tuner = tune.Tuner() // results = tuner.fit() Code example? I’ve tried to pass the server_port as an argument to the Tuner Object, to the RunConfig() or to the TuneConfig() but there was always an error regarding an unexpected argument. I’ve also seen code, where the argument with_server=True was passed to the tune.run() call, but also this didn’t work for me trying to pass it to the Tuner Object.
Maybe someone knows how to achieve the same behaviour as with tune.run().
Thanks in advance for your help!