Saving Gym Environment Video with RLlib

RLlib
1

Hi! I’m trying to save videos of the training process on the Gym Car Racing environment using the Python API for RLlib. Here’s my code:

import gym
import ray
from gym.wrappers import ResizeObservation
from ray.tune.registry import register_env

from ray import tune
import ray.rllib.agents.ppo as ppo

def env_creator(env_config):
    env = gym.make('CarRacing-v0', **env_config)
    return ResizeObservation(env, (84, 84))

ray.shutdown()
ray.init(ignore_reinit_error=True)

register_env('CarRacingResized', env_creator)

ENV = 'CarRacingResized'
CHECKPOINT = './ray/racing'
config = ppo.DEFAULT_CONFIG.copy()
config['log_level'] = 'INFO'
config['framework'] = 'torch'
config['monitor'] = True

agent = ppo.PPOTrainer(config, env=ENV)

N_ITER = 30
s = '{:3d} reward {:6.2f}/{:6.2f}/{:6.2f} len {:6.2f} saved {}'

for n in range(N_ITER):
  result = agent.train()
  file_name = agent.save(CHECKPOINT)

  print(s.format(
    n + 1,
    result['episode_reward_min'],
    result['episode_reward_mean'],
    result['episode_reward_max'],
    result['episode_len_mean'],
    file_name
   ))

However, the videos that get saved in the ~/ray_results directory are just a black screen for a split second. I’ve also tried saving videos of the CartPole environment and I ran into the same issue. I thought that the config['monitor'] flag was all I needed to save video. Am I doing something incorrectly?

Thank you!