[Tune] Wandb video recording integration is broken

Ray Tune
1

Hi folks,

I’ve been working on setting up a simple test script for integrating wandb and I’ve run into an issue with wandb’s monitor_gym setting. If this is set to True then gym’s Monitor wrapper is modified so any videos that are created will automatically be uploaded to wandb. My theory is that this has something to do with each rllib worker loading its own libraries. I’m not yet sure what the best approach is to fix this, but it would be nice if this could automatically be taken care of when the WandbLoggerCallback is used.

Here’s my test script if you want to quickly reproduce this:

tune.run(
    "PPO",
    config={
        "env": "CartPole-v0",
        "num_workers": 1,
        "num_envs_per_worker": 1,
        "record_env": "videos"
    },
    callbacks=[
        WandbLoggerCallback(
            project="project_name",
            api_key_file="api_key.txt"
            monitor_gym=True # calls wandb.gym.monitor() -> gym's Monitor wrapper is modified to automatically log videos
        )
    ],
    stop={
        "training_iteration": 1
    }
)
1 Like
3

Hey @RickDW , I think the problem is that RLlib has its own monitor wrapper (a slightly different one from gym’s original monitor wrapper), which may conflict with WanDB’s wrapper?

4

I’m not using a multi-agent environment, so gym’s own Monitor wrapper should be used unless I’m misunderstanding something (?) I’ve been digging through rllib’s source and found the function record_env_wrapper in rllib.env.utils, which uses the standard gym Monitor class. This is the function that is used in the RolloutWorker class so I figured there is no custom env involved. Could you clear this up for me?