Change config before trial starts

Ray Libraries (Data, Train, Tune, Serve) Ray Tune
1

How severe does this issue affect your experience of using Ray?

  • Medium: It contributes to significant difficulty to complete my task, but I can work around it.

I need a specific config for each trial. So I have defined a custom callback and overrided on_trial_start with:

def on_trial_start(self, iteration: int, trials: List["Trial"],
                       trial: Trial, **info):
     config = deepcopy(trial.config)
     config["some_key_I_want_to_change"] = "new value"
     trail.set_config(config)

but it does not work. i.e. the trainable does not receive the updated config.

I found a workaround but it is messy. First I define a callback function in my config.

def update_trial_config(config, trainable_name, trial_id):
     config["some_key_I_want_to_change"] = "new value"

config["update_trial_config"] = update_trial_config

then in the file trial.py I call this callback function. To do so, in the __init__ function of Trial I replace the line self.config = config or {} by:

self.config = config or {}
if "update_trial_config" in self.config:
    self.config = self.config["update_trial_config"](self.config, self.trainable_name, self.trial_id)

What is the right way to do this?

2

Hi @Nicolas_Carrara, updating the config so that it’s reflected in e.g. the Ray Tune reporting is not straightforward. Could you elaborate a bit on your use case so we can help find a good solution?

It would be best to write your config in a way that you don’t have to overwrite parameters later. There are a few ways to do this.

One way is grid search, which ensures that certain values are set:

config={
    "a": tune.grid_search([1, 2, 3]),
    "b": tune.randint(0, 10)
}

this will create three trials with the specified values of a (a=1, a=2, a=3) and a random value for b.

Another way is to use conditional parameters that can depend on other parameters:

    config={
        "a": tune.randint(0, 10),
        "b": tune.sample_from(lambda spec: spec.config.a * 2)
    }
3

Hi @kai , thanks for you reply.

Playing with grid_search and sample_from won’t help, because it will actually create more trials. This is exactly my problem: for each trial created AFTER the resolved config, I need to do some processing specific to this very trial, with this very params resolution.

This how it works:

  • I have a web server that creates generators of trajectories (episodes). The server keeps a pool of those generators.
  • Each environment is associated with a generator, in order to generate a new trajectory after each reset(). A generator can be specific to be environment or share among workers.
  • I must specify in the env_config, what is the url to call in order to call next() on the generator (for example localhost:8888/generator1/next). This url will depends on the resolution of the original configuration.
  • Each trial created after the resolution of the original configuration must then ask the server to create a new generator. The server then sends back to the trial the URL to call for calling next() on the generator.

I need this before_trial_start callback because I cannot know the URL before the config is fully resolved.

4

Could you maybe use tune.sample_from for this (you can pass a full function there)?

5

You suggest creating all the possible combinations of grid-searched parameters given the configuration file. Then associate with each combination a unique ID using a tune.sample_from conditioned on the combination?

What is wrong with a callback just before starting a trial? Does it have unexpected implications? From what I see, Tune will save the resolved config with the specific parameters of each trial, so why not adding extra parameters before the trial start?

6

I don’t know the constraints from your system, but the sample_from should only be called when the trial configuration is generated, so you can use this to lazy-initialize parameter values.

The callback is executed on the driver and the configuration is not passed to the trainable again. The contract here is that we have specific APIs where configurations can change, so that the trainable can assume that there are no changes otherwise.

What you could do though is to request the URl from within the trainable (i.e. in the training function). If you want to log this in the results, you can just report it as a metric in tune.report(). Would that work?

7

Thanks for the explanation!

I think sample_from should do the trick if it is called after the configuration is resolved!