[RLlib] when calling trainer.eval(), the program is blocked

RLlib
1

How severe does this issue affect your experience of using Ray?

  • High: It blocks me to complete my task.

Hi, I was trained an agent, but the program was blocked when I was calling trainer.eval()
This is my code:

config = {
        'env': 'maze_env',
        'gamma': 0.99,
        'framework': 'torch',
        'num_workers': 1,
        'model': {
            'custom_model': 'maze_model',
            'custom_action_dist': 'maze_model'
        },
        'explore': False,
        'evaluation_interval': 1,
        'evaluation_duration': 1,
    }
    trainer = ppo.PPOTrainer(config=config, env='maze_env')
    trainer.restore('./trained_models/maze_v2/checkpoint_000001/checkpoint-1')
    res = trainer.evaluate()

After running, the program is in the following state and stops (for a long time)

2022-04-15 15:17:03,025	WARNING util.py:55 -- Install gputil for GPU system monitoring.
2022-04-15 15:17:03,047	INFO trainable.py:496 -- Restored on 127.0.0.1 from checkpoint: ./trained_models/maze_v2/checkpoint_000001/checkpoint-1
2022-04-15 15:17:03,047	INFO trainable.py:503 -- Current state after restoring: {'_iteration': 1, '_timesteps_total': 4000, '_time_total': 11.056960105895996, '_episodes_total': 30}

I want to know which part of me is wrong.
What are the best practices for config of trainer.evaluate()