# RLlib callbacks to get custom metrics such as observation, reward...etc in each episode from SingleAgentEpisode and access it in the trainer

**URL:** <https://discuss.ray.io/t/rllib-callbacks-to-get-custom-metrics-such-as-observation-reward-etc-in-each-episode-from-singleagentepisode-and-access-it-in-the-trainer/16022>\
**Category:** RLlib\
**Created:** [October 17, 2024, 8:10am UTC](https://discuss.ray.io/t/rllib-callbacks-to-get-custom-metrics-such-as-observation-reward-etc-in-each-episode-from-singleagentepisode-and-access-it-in-the-trainer/16022 "2024-10-17T08:10:44Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![varnie](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/varnie/32/6648_2.png) [@varnie](https://discuss.ray.io/u/varnie)\
**Post date:** [October 17, 2024, 8:10am UTC](https://discuss.ray.io/t/rllib-callbacks-to-get-custom-metrics-such-as-observation-reward-etc-in-each-episode-from-singleagentepisode-and-access-it-in-the-trainer/16022/1 "2024-10-17T08:10:45Z")

</div>

As part of the project I am working now I need to be able get access to intermediary values such as observations, reward, actions, action probability for each episode and access it in the result dictionary returned by the trainer class when we call .train().  
Before SingleAgentEpisode in the new api stack, i.e. EpisodeV2, we were able to access these data like below:

```auto
def on_episode_end(self, *, episode: SingleAgentEpisode, **kwargs):
        actions = episode.actions.data
        observations = episode.observations.data
        rewards = episode.rewards.data
        episode.custom_metrics["observations"] = observations
        episode.custom_metrics["actions"] = actions
        episode.custom_metrics["rewards"] = rewards

```

But SingleAgentEpisode no longer has the attribute custom metrics which means I can’t access these data in the result. Does anyone know how to access these in the new api stack?

---

<div class="post-metadata">

**Author:** ![hendrikunger](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/hendrikunger/32/7315_2.png) [@hendrikunger](https://discuss.ray.io/u/hendrikunger)\
**Post date:** [November 14, 2024, 8:55am UTC](https://discuss.ray.io/t/rllib-callbacks-to-get-custom-metrics-such-as-observation-reward-etc-in-each-episode-from-singleagentepisode-and-access-it-in-the-trainer/16022/2 "2024-11-14T08:55:11Z")

</div>

I have exactly the same problem. I found a partial solution using the MetricsLogger available in the new callbacks.  
The problem now is that on\_episode\_start, on\_episode\_step and on\_episode\_end, where I need to collect my metrics, returns the MetricsLogger on the env\_runner.

I need to do my calculations and preparations for the wandb upload in on\_evaluate\_end.  
But in on\_evaluate\_end the MetricsLogger object of the algorithm class is returned.  
If this would hold the objects I logged in the on\_episode\_step callback, everything would be fine. But the metrics seem to be reduced even if reduce=None is set while logging the metrics in on\_episode\_step.

So maybe this gives you a further clue or somebody with more knowledge can step in and explain how it is supposed to work.

---

<div class="post-metadata">

**Author:** ![hendrikunger](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/hendrikunger/32/7315_2.png) [@hendrikunger](https://discuss.ray.io/u/hendrikunger)\
**Post date:** [November 19, 2024, 9:35am UTC](https://discuss.ray.io/t/rllib-callbacks-to-get-custom-metrics-such-as-observation-reward-etc-in-each-episode-from-singleagentepisode-and-access-it-in-the-trainer/16022/3 "2024-11-19T09:35:09Z")

</div>

Solved the problem:

You can using the MetricsLogger in any of the callbacks

```auto
metrics_logger.log_value(("myData",infos[0].get('evalEnvID', 0)+1), infos, reduce=None, clear_on_reduce=True)

```

Then you can get back your data in the on\_evaluate\_end callback with

```auto
data = evaluation_metrics["env_runners"]["myData"]

```

.  
I think it will be reduced unless you specifiy reduce=None during logging.