When running multiple RL experiments with evaluation during training, rllib reports the evaluation metrics to tensorboard as “ray/tune/evaluation/…”.
How is it possible to access those metrics programmatically for analysis after training?
I looked into
tune.Analysis to easily get statistics about multiple experiments, it works great but it has everything except the evaluation data
import ray.tune as tune EXPERIMENT_FOLDER = "/home/username/ray_results/my_experiments" analysis = tune.Analysis(EXPERIMENT_FOLDER, default_metric="episode_reward_mean", default_mode="max") df = analysis.dataframe() for c in df.columns: print(c)
This will print:
episode_reward_max episode_reward_min episode_reward_mean episode_len_mean episodes_this_iter ... custom_metrics/... ... config/...
I am looking for a similar solution to get evaluation data. Thanks
Using Ray version 1.2.0.