I’m moving this Slack discussion over here, so I’ll try to wrap it as clear and short as possible for future references. What I want: A metric over evaluation episodes to save and load checkpoints based on that score. Eg: Train some agent to play Flappy birds, => Evaluate while training using a…

Thanks for asking this question. Could you create a small reproduction script(s), so I can more easily debug this?

[image] sven1977: Could you create a small reproduction script(s), so I can more easily debug this? Hi Sven! I have been writing a notebook these days so the “issue” can be reproduced (and maybe later on write some kind of notebook tutorial example on this). I created a simple environment wh…

@LecJackS @sven1977 I am curious to know a solution to this since I appear to have the same issue. I have a custom metric “my_metric” which I have defined “on_episode_end” and would like to use checkpoint_score_attr=‘custom_metrics.my_metric_mean’ in tune.run(). However, I receive the error …

@GallantWood Couldn’t make it work in my case. I just run several experiments for a large number of iteration, save checkpoints by iteration, and then choose the best checkpoint using some plots and statistics I run later on for each saved checkpoint.

@LecJackS @GallantWood Have a look at this topic that might be relevant [image] Saving checkpoints with good custom_metric using tune.run() Ray Tune Hi @mannyv Thank you for helping. I have been running Ray 1.2. I repeated test with 1.3 and was able to reproduce your res…

Thank you @mannyv That was helpful. It looks like the changes in the thread you linked to have been implemented in Ray ver 1.9 (which I am now using). Since my issue remains, I assume I need to follow your final response which is to “explicitly request the json in the config.”. Can you elaborate …

I have discovered that my particular issue has to do with some idiosyncratic behavior involving the interaction between “episode horizon” and checkpoint saving in RLlib. Therefore, if I can not resolve the problem, I will open a new issue specific to this.

Custom metrics over evaluation only

RLlib

mannyv December 3, 2021, 3:27pm 8

@LecJackS @GallantWood

Have a look at this topic that might be relevant

Topic		Replies	Views
Saving checkpoints with good custom_metric using tune.run() Ray Tune	18	2366	July 20, 2021
Use `checkpoint_score_attr` with custom metric Ray Tune	3	534	May 11, 2022
Store best checkpoints according to evaluation metrics Checkpointing, Restoring	0	392	June 19, 2023
[Rllib, Tune, AIR] Checkpointing as per custom metric minimum Checkpointing, Restoring	5	94	July 2, 2025
Custom metric that combines train and eval results RLlib	2	461	June 10, 2022

Custom metrics over evaluation only

Related topics