ICM - Curiosity Reward Scale

Hi all!

I am curious if there is way to see the value of the reward from the curiosity module? Because I am currently utilising it and feel as though the scale of my extrinsic and intrinsic (curiosity) rewards are completely out of sync.

Thanks in advance!

I am using PPOTrainer with the Policy Server+Client set-up

1 Like

I have the same question, did you find out a way to do so?

I ended up abondining curiosity (not because anything was wrong with it) but it just was the wrong tool for my use case, like really bad lol.

That being said, I later discovered you could get really, really nice graphs auto-generated if you setup your training loop kinda like this:

         # resume = True,
         config=DEFAULT_CONFIG, name=name, keep_checkpoints_num=None, checkpoint_score_attr="episode_reward_mean",
         # restore="C:\\Users\\-----\\ray_results\\TEST_32k-batch_512-len_32_Run-2\\PPO_RandomEnv_46610_00000_0_2021-12-31_17-30-37\\checkpoint_000027\\checkpoint-27",
         checkpoint_freq=1, checkpoint_at_end=True)

Then you go to localhost:6006. You will see a breakdown of rewards, episode lenghs, and all other data. I’m guessing you can find curisiosity rewards in there as well.

If you end up trying, let me know!

Hi @Denys_Ashikhin ,

If you what you are looking at is at port 6006, it’s probably tensorboard.
Here are the related docs.