ICM - Curiosity Reward Scale

RLlib
1

Hi all!

I am curious if there is way to see the value of the reward from the curiosity module? Because I am currently utilising it and feel as though the scale of my extrinsic and intrinsic (curiosity) rewards are completely out of sync.

Thanks in advance!

P.S.
I am using PPOTrainer with the Policy Server+Client set-up

1 Like
2

I have the same question, did you find out a way to do so?

3

I ended up abondining curiosity (not because anything was wrong with it) but it just was the wrong tool for my use case, like really bad lol.

That being said, I later discovered you could get really, really nice graphs auto-generated if you setup your training loop kinda like this:

tune.run(trainer,
         # resume = True,
         config=DEFAULT_CONFIG, name=name, keep_checkpoints_num=None, checkpoint_score_attr="episode_reward_mean",
         max_failures=1,
         # restore="C:\\Users\\-----\\ray_results\\TEST_32k-batch_512-len_32_Run-2\\PPO_RandomEnv_46610_00000_0_2021-12-31_17-30-37\\checkpoint_000027\\checkpoint-27",
         checkpoint_freq=1, checkpoint_at_end=True)

Then you go to localhost:6006. You will see a breakdown of rewards, episode lenghs, and all other data. I’m guessing you can find curisiosity rewards in there as well.

If you end up trying, let me know!

4

Hi @Denys_Ashikhin ,

If you what you are looking at is at port 6006, it’s probably tensorboard.
Here are the related docs.