Why does tune.run(config=tune.grid_search(...)) work?

Ray Libraries (Data, Train, Tune, Serve) Ray Tune
1

I have a problem that requires me to run a Tune experiment with multiple ready-made configurations. In doing so, I have fortunately stumbled upon the fact that the call tune.run(some_function, config=tune.grid_search([config0, config1, ...])
works and I would really like to know why. The signature of the function looks like this:

@PublicAPI
def run(
        run_or_experiment: Union[str, Callable, Type],
        config: Optional[Dict[str, Any]] = None, 
        ...
)

I do fulfill the signature of the method with the given pass, because

tune.grid_search([...])

leads to the following expression:

{
    "grid_search": [...]
}

which is basically a dictionary with a string as key but I would like to know, why the subsequent framework can also handle it.

2

Hey @LukasNothhelfer,

This is because the grid_search key is being resolved by variant_generator.py and will generate individual variants!

As an example, let’s take a look if we use tune.grid_search to define a key a in config:

config={"a": tune.grid_search([1, 2])}

config={"a": {"grid_search": [1, 2]}}

# Two variants are generated:
{"a": 1}
{"a": 2}

Notice that {"grid_search": values_list}} will be replaced with a value for each of the len(values_list) variants generated.

We can apply the same logic to see why this works when using tune.grid_search at the top level:

config=tune.grid_search([{"a": 1}, {"a": 2}])

config={"grid_search": [{"a": 1}, {"a": 2}]}

# Two variants are generated:
{"a": 1}
{"a": 2}

Implementation-wise, the unresolved_spec that is passed into generate_variants is a Dict which has "config" as a key, so the same logic is applied at this top level as it is for nested keys.