Accessing custom metrics for episodes


I’ve used callbacks before to measure custom metrics on episodes. The way I usually
do it is by implementing the on_episode_end method, and adding a line like this:

(env,) = base_env.get_sub_environments()
episode.custom_metrics |= {
    'my_metric': env.my_metric,

This works, and then later, Rllib adds an element my_metric_mean to the result, which my code reads.

What’s different now is that I have a metric that I requires a weighted average with some weight that I define. This means that I can’t let RLlib do the averaging for me. I want my code to somehow receive the my_metric value for each of the episodes that ran.

How do I do this? How do I get access to this metric from all the episodes in the evaluation?

Thanks for your help,
Ram Rachum.

Oh, I just found keep_per_episode_custom_metrics=True, so this solves my problem.