Grouping variables together in ray tune

Ray Libraries (Data, Train, Tune, Serve) Ray Tune
1

I have a ray tune configuration which is specified as follows:


config={
         "params.central_layer_sizes:": tune.grid_search([[1600,800], [403, 200], [400, 400]]),
         "params.right_layer_sizes": tune.grid_search([[48,24], [12, 6], [12, 12]]),
         "params.joint_layer_sizes": tune.grid_search([[800,400], [400, 200], [400, 400]])
     }

I would like to sample in a concerted way i.e. the index 0 of the three variables go together, index 1 and index 2. Is there a way to specify such grouping out of the box in ray? At the moment, everything is sampled independent of the other and generating configurations that I do not want to test.

2

got it, could you try grouping them before passing into config and handle it in your function?

Something like:

tune.grid_search([[[1600,800], [48,24], [800,400]], [...], [...]])
3

The thing is I am using Facebook hydra for specifying all the configurations and it is a nice way to somehow pass directly from config to the object instantiation but I am guessing there is no way to bypass some custom mapping code in the middle.

4

@pamparana, I don’t know, if this helps with your problem, but with tune.sample_from() you can actually define your custom sample functions using for example a lambda function as shown here for PyTorch training.