Loading experiment analysis from a different machine than the experiment was run with

Ray Libraries (Data, Train, Tune, Serve) Ray Tune
1

hello, I am training a PPO agent with RLlib in one machine using windows and then I copy the experiment folder to a different machine with Linux for testing purposes.

To get the analysis object I perform the following operation hoping to be able to get get_best_trial() and get_best_checkpoint() and thenafter be able to build the PPO algorithm from the checkpoint:

analysis_object = ExperimentAnalysis(Linux_experiment_path,
                                         default_metric=metric, 
                                         default_mode=mode)

However, analysis_object path always refers to the original windows path producing errors.

What is the proper workflow in this case?

1 Like
2

same issue!
Please let me know if you find a solution.

3

This is a known issue that’s being tracked here: [Train/Tune] Restore an experiment from a different machine/path · Issue #40585 · ray-project/ray · GitHub

Targeting a fix for Ray 2.9, but will keep this thread updated if a nightly is available earlier for you to use. Thanks for raising this issue.

4

Thank you very much @justinvyu
I am glad is something identified.
I have been trying to solve this for days before advancing since If I started to train models on the remote machine I could not load checkpoints on my local machine and test analyse, etc

1 Like
5

This should be fixed in the nightly version of ray by this PR: [tune/train] Restore Tuner and results properly from moved storage path by justinvyu · Pull Request #40647 · ray-project/ray · GitHub.

https://docs.ray.io/en/latest/ray-overview/installation.html

Let me know if you get the chance to try it out!

6

Thank you very much. As soon as I try I will inform here.
Best

1 Like