IMPALA agent not working

RLlib
1

Hey everybody, so I’m using the rayproject/ray-ml:b42155-py310-cu116 docker image and I’m trying to run the IMPALA agent as described in the documentation RLlib IMPALA documentation

from ray.rllib.algorithms.impala import ImpalaConfig
config = ImpalaConfig()
config = config.training(lr=0.0003, train_batch_size=512)  # doctest: +SKIP
config = config.resources(num_gpus=4)  # doctest: +SKIP
config = config.rollouts(num_rollout_workers=64)  # doctest: +SKIP
print(config.to_dict())  # doctest: +SKIP
# Build a Algorithm object from the config and run 1 training iteration.
algo = config.build(env="CartPole-v1")  # doctest: +SKIP
algo.train()  # doctest: +SKIP

When I run this with 1 gpu (changing num_gpus=1) It crashes
INVALID_ARGUMENT: Input to reshape is a tensor with 512 values, but the requested shae has 500
Does anyone have any idea why is this not working out of the box ?

2

Hi @aea,

The docstring example codes are not guaranteed to work right of the box as of now since they are not doc-tested. If you don’t provide train_batch_size to leave the default 500, your code would work. Basically:

from ray.rllib.algorithms.impala import ImpalaConfig
config = ImpalaConfig()
config = config.training(lr=0.0003)  # doctest: +SKIP
config = config.resources(num_gpus=1)  # doctest: +SKIP
config = config.rollouts(num_rollout_workers=64)  # doctest: +SKIP
print(config.to_dict())  # doctest: +SKIP
# Build a Algorithm object from the config and run 1 training iteration.
algo = config.build(env="CartPole-v1")  # doctest: +SKIP
algo.train()  # doctest: +SKIP