# Custom Tensorboard Metric (episode.total\_reward auto generates as mean, min, max) **URL:** \ **Category:** RLlib\ **Created:** [June 11, 2024, 8:45pm UTC](https://discuss.ray.io/t/custom-tensorboard-metric-episode-total-reward-auto-generates-as-mean-min-max/14944 "2024-06-11T20:45:41Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![VisionZUS29](https://avatars.discourse-cdn.com/v4/letter/v/ea666f/32.png) [@VisionZUS29](https://discuss.ray.io/u/VisionZUS29)\ **Post date:** [June 11, 2024, 8:45pm UTC](https://discuss.ray.io/t/custom-tensorboard-metric-episode-total-reward-auto-generates-as-mean-min-max/14944/1 "2024-06-11T20:45:41Z")
**How severe does this issue affect your experience of using Ray?** - Low: It annoys or frustrates me for a moment. I want to monitor the episode reward throughout training, rather than seeing min, max, mean plots. I used this callback and my tensorboard is updated with the custom metric, but it automatically creates mean, min, max instead of just episode reward itself. ray/tune/custom\_metrics/individual\_episode\_reward\_mean ray/tune/custom\_metrics/individual\_episode\_reward\_min ray/tune/custom\_metrics/individual\_episode\_reward\_max Printing episode.total\_reward to terminal during training gives a singular scalar value per episode as I would expect. ```auto class CustomCallbacks(DefaultCallbacks): def on_episode_end(self, *, worker, base_env, policies, episode, **kwargs): # Log the total reward for the episode\ print("Episode Reward: ", episode.total_reward) episode.custom_metrics["individual_episode_reward"] = episode.total_reward ``` I want basically what is in ray/tune/hist\_stats/episode\_reward but as a scalar instead of histogram plot. ---
**Author:** ![mannyv](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/mannyv/32/606_2.png) [@mannyv](https://discuss.ray.io/u/mannyv)\ **Post date:** [June 12, 2024, 1:30am UTC](https://discuss.ray.io/t/custom-tensorboard-metric-episode-total-reward-auto-generates-as-mean-min-max/14944/2 "2024-06-12T01:30:52Z")
Hi @VisionZUS29, The reason it reports a mean, min, max is because during one iteration it is potentially sampling transition s from multiple episodes. If multiple episodes complete then there will be multiple episode returns and the metrics report the min, max, mean of all the episodes. Hist stats shows you a histogram of every episode return obtained in an iteration. ---
**Author:** ![VisionZUS29](https://avatars.discourse-cdn.com/v4/letter/v/ea666f/32.png) [@VisionZUS29](https://discuss.ray.io/u/VisionZUS29)\ **Post date:** [June 12, 2024, 2:07am UTC](https://discuss.ray.io/t/custom-tensorboard-metric-episode-total-reward-auto-generates-as-mean-min-max/14944/3 "2024-06-12T02:07:49Z")
@mannyv What do you mean by iteration, Is it sampling based on whether the config has collections as complete\_episodes vs truncate\_episodes? I guess then i am looking to be able to plot reward vs episode rather than steps? My env terminates after a set amount of steps so each episode has the same amount of steps. I still think it should make sense to have a singular episode reward plot. ---
**Author:** ![mannyv](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/mannyv/32/606_2.png) [@mannyv](https://discuss.ray.io/u/mannyv)\ **Post date:** [June 12, 2024, 2:25am UTC](https://discuss.ray.io/t/custom-tensorboard-metric-episode-total-reward-auto-generates-as-mean-min-max/14944/4 "2024-06-12T02:25:55Z")
Perhaps this thread will be helpful: > [@Meaning of episode\_reward\_mean](https://discuss.ray.io/t/meaning-of-episode-reward-mean/3839/1): > > What is the meaning of the episode\_reward\_mean metric? Is it the sum of the reward obtained in each time step of the episode? What is the difference between episode\_reward\_mean and episode\_reward\_min? ---
**Author:** ![VisionZUS29](https://avatars.discourse-cdn.com/v4/letter/v/ea666f/32.png) [@VisionZUS29](https://discuss.ray.io/u/VisionZUS29)\ **Post date:** [June 12, 2024, 3:43am UTC](https://discuss.ray.io/t/custom-tensorboard-metric-episode-total-reward-auto-generates-as-mean-min-max/14944/5 "2024-06-12T03:43:01Z")
@mannyv I get the gist of what you are saying. I understand the difference in mean, min, max. > [@Meaning of episode\_reward\_mean](https://discuss.ray.io/t/meaning-of-episode-reward-mean/3839/4): > > RLLIB collects a number of metrics. One of those is the episode\_reward. When creating the summary it will compute and store the mean, min , and max of those metrics. > The metrics summarize all of the collected values during the previous iteration. What defines an iteration varies. If you are using PPO, for example, the episode reward will contain the rewards obtained for every completed episode that occurred when collecting the last `train_batch_size` steps. If no episodes returned done which is possible for some really long environments, then that metric would be empty. So why can’t I plot every episode reward for every completed episode that occurred throughout the training? If it’s already being collected and sorted per train\_batch\_size? ---
**Author:** ![narevau](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/narevau/32/5799_2.png) [@narevau](https://discuss.ray.io/u/narevau)\ **Post date:** [June 24, 2024, 4:06pm UTC](https://discuss.ray.io/t/custom-tensorboard-metric-episode-total-reward-auto-generates-as-mean-min-max/14944/6 "2024-06-24T16:06:41Z")
Hi @VisionZUS29, not quite sure if this is what you want, but i think you can set it in “ **episode.hist\_data** ” like this: ```auto def on_episode_end(self, *, worker, base_env, policies, episode, **kwargs): # Log the total reward for the episode\ print("Episode Reward: ", episode.total_reward) episode.hist_data["individual_episode_reward"] = episode.total_reward ``` It should then keep track of every reward seen in each episode in the result of: result = algo.evaluate(), somewhere under hist\_data.