I am running an experiment with
tuner = tune.Tuner(trainable_with_resources, param_space=config, tune_config=tune_config, run_config=run_config)
results = tuner.fit()
That’s all nice, but what if I want to add trials to my experiment later, such that I can do something like
analysis = ExperimentAnalysis((run_config.storage_path)
for all trials within run_config.storage_path?
- If I just start a new experiment and save it into run_config.strorage_path, only the new trials will be read for analysis.
- Tuner.restore is only meant for incomplete experiments
I have not found a solution that works for me, yet…Thank you!
Hi @Jaydeen , do you mean you want to save two experiment results into the same storage path? May I know your use case?
Currently ExperimentAnalysis doesn’t support it. For result analysis, maybe you can merge multiple
result.json files in different experiment folders.
Hi! Thanks for your reply! Say I conducted a grid_search for parameters A and B with the param_space:
At a later time I want to add the option B = 6 to my search space. How would I do that? Yes, I can run a new experiment with
but if I want to analyse my results later I would somehow need to merge the results manually. It would be nice if I could just add the runs to the original experiment folder …
Hope that was clearer? Thank you!
Got it. Thank you @Jaydeen. Currently Ray Tune doesn’t support this feature yet, you have to manually merge the result files and do some post-process.
I think this is a reasonable use case, we’ll discuss internally whether to support this feature.