Accessing custom metrics for episodes

RLlib
1

Hi,

I’ve used callbacks before to measure custom metrics on episodes. The way I usually
do it is by implementing the on_episode_end method, and adding a line like this:

(env,) = base_env.get_sub_environments()
episode.custom_metrics |= {
    'my_metric': env.my_metric,
}

This works, and then later, Rllib adds an element my_metric_mean to the result, which my code reads.

What’s different now is that I have a metric that I requires a weighted average with some weight that I define. This means that I can’t let RLlib do the averaging for me. I want my code to somehow receive the my_metric value for each of the episodes that ran.

How do I do this? How do I get access to this metric from all the episodes in the evaluation?

Thanks for your help,
Ram Rachum.

2

Oh, I just found keep_per_episode_custom_metrics=True, so this solves my problem.

1 Like
3

Where do you use this configuration?

4

@AlyDelas you can find it here: ray.rllib.algorithms.algorithm_config.AlgorithmConfig.reporting — Ray 2.9.3
Unfortunately all of your custom_metrics will now be per episode. You cant specify per metric.