Accessing param space after restore

Hi All :wave:

Iโ€™m new to RayTune and was trying to analyse a hyperparameter run. I am able to load the tuner object but the param space is empty, ie this restored tuner has no idea of the parameter space used for the tuning.
Is there a way for the Tuner to save the param_space values? Because everything else is being saved, why not the parameter search space as well?

Best :slight_smile:

@f2010126 one way to persist parameter space and configs and re-ruse them is to use MLflow in conjunction with Ray Tune.

cc: @kai @justinvyu Is there another way to persist and extract parameter_space?

@Jules_Damji Thanks so much for the suggestion! But it would not really help when the experiment is done and all I have are the generated artefacts and logs. Also I wanted to use only the RayTune functionalities for now.

Is there some way to add the option (as a new feature, maybe) to save the config space when logging other run related files? Eg for when you want to reproduce someone elseโ€™s HPO runs.

Best :slight_smile:

Hi @f2010126,

how do you restore the tuner object?

Generally the searcher stored in the experiment directory (which is also loaded with the tuner) implicitly contains the parameter space, but it will be hard to extract it. We require the param space to be overwritten on restore because it can contain ray objects that will not exist when loaded in a different ray session.

It would be great to understand the specific use case here - to reproduce runs, can the user just post their param space definition somewhere? Using someone elses stored Tuner can lead to some downstream problems anyway (e.g. depending on specific package versions used).