Error running intro code

RLlib
1

How severe does this issue affect your experience of using Ray?

  • High: It blocks me to complete my task.

I’m new to Ray and RLlib and thought I’d start at the beginning, but I cannot run the first block of Python code (copied and pasted from Getting Started with RLlib — Ray 2.9.3)

from ray.rllib.algorithms.ppo import PPOConfig
from ray.tune.logger import pretty_print


algo = (
    PPOConfig()
    .rollouts(num_rollout_workers=1)
    .resources(num_gpus=0)
    .environment(env="CartPole-v1")
    .build()
)

for i in range(10):
    result = algo.train()
    print(pretty_print(result))

    if i % 5 == 0:
        checkpoint_dir = algo.save().checkpoint.path
        print(f"Checkpoint saved in directory {checkpoint_dir}"

it throws the error as follows:
– Ray error, taking actor 1 out of service. The actor died because of an error raised in its creation task, ray::RolloutWorker.init() (pid=17364, ip=127.0.0.1, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x000002855DDF04F0>)
ValueError: The two structures don’t have the same nested structure.
First structure: type=tuple str=(array([ 0.0126949 , -0.03816216, -0.04357562, 0.02119169], dtype=float32), {})

Second structure: type=ndarray str=[-4.7838573e+00 2.1734766e+38 2.6380509e-01 3.1755948e+38]

More specifically: Substructure “type=tuple str=(array([ 0.0126949 , -0.03816216, -0.04357562, 0.02119169], dtype=float32), {})” is a sequence, while substructure “type=ndarray str=[-4.7838573e+00 2.1734766e+38 2.6380509e-01 3.1755948e+38]” is not

I am surprised that this is not working as it’s such a simple piece of intro code - any suggestions as to why?

I’m running this on Windows 10 using Thonny IDE

2

@Paul_Vanags I am sorry to hear that you run into problems. I cannot reproduce this on master. COuld you try to update to the newest version and retry?

1 Like
3

Ah OK - I was on Ray 2.2, upgrading to 2.9.x fixed it thank you

1 Like