'AgentId' object has no attribute 'shape'

RLlib
1

Hello,

I am trying to migrate from Ray 0.8.7 to Ray 1.1.0.

One of the issue that I am encountering when doing a simple PPO tune.run is the following:

Error processing event.
Traceback (most recent call last):
  File "/home/faten/anaconda3/envs/X/lib/python3.8/site-packages/ray/tune/trial_runner.py", line 519, in _process_trial
    result = self.trial_executor.fetch_result(trial)
  File "/home/faten/anaconda3/envs/X/lib/python3.8/site-packages/ray/tune/ray_trial_executor.py", line 497, in fetch_result
    result = ray.get(trial_future[0], timeout=DEFAULT_GET_TIMEOUT)
  File "/home/faten/anaconda3/envs/X/lib/python3.8/site-packages/ray/worker.py", line 1379, in get
    raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(AttributeError): ray::PPO.train() ()
  File "python/ray/_raylet.pyx", line 463, in ray._raylet.execute_task
  File "python/ray/_raylet.pyx", line 415, in ray._raylet.execute_task.function_executor
  File "/home/faten/anaconda3/envs/X/lib/python3.8/site-packages/ray/rllib/agents/trainer.py", line 505, in train
    raise e
  File "/home/faten/anaconda3/envs/X/lib/python3.8/site-packages/ray/rllib/agents/trainer.py", line 494, in train
    result = Trainable.train(self)
  File "/home/faten/anaconda3/envs/X/lib/python3.8/site-packages/ray/tune/trainable.py", line 183, in train
    result = self.step()
  File "/home/faten/anaconda3/envs/X/lib/python3.8/site-packages/ray/rllib/agents/trainer_template.py", line 147, in step
    res = next(self.train_exec_impl)
  File "/home/faten/anaconda3/envs/X/lib/python3.8/site-packages/ray/util/iter.py", line 756, in __next__
    return next(self.built_iterator)
  File "/home/faten/anaconda3/envs/X/lib/python3.8/site-packages/ray/util/iter.py", line 783, in apply_foreach
    for item in it:
  File "/home/faten/anaconda3/envs/X/lib/python3.8/site-packages/ray/util/iter.py", line 783, in apply_foreach
    for item in it:
  File "/home/faten/anaconda3/envs/X/lib/python3.8/site-packages/ray/util/iter.py", line 843, in apply_filter
    for item in it:
  File "/home/faten/anaconda3/envs/X/lib/python3.8/site-packages/ray/util/iter.py", line 843, in apply_filter
    for item in it:
  File "/home/faten/anaconda3/envs/X/lib/python3.8/site-packages/ray/util/iter.py", line 783, in apply_foreach
    for item in it:
  File "/home/faten/anaconda3/envs/X/lib/python3.8/site-packages/ray/util/iter.py", line 783, in apply_foreach
    for item in it:
  File "/home/faten/anaconda3/envs/X/lib/python3.8/site-packages/ray/util/iter.py", line 783, in apply_foreach
    for item in it:
  [Previous line repeated 1 more time]
  File "/home/faten/anaconda3/envs/X/lib/python3.8/site-packages/ray/util/iter.py", line 876, in apply_flatten
    for item in it:
  File "/home/faten/anaconda3/envs/X/lib/python3.8/site-packages/ray/util/iter.py", line 828, in add_wait_hooks
    item = next(it)
  File "/home/faten/anaconda3/envs/X/lib/python3.8/site-packages/ray/util/iter.py", line 783, in apply_foreach
    for item in it:
  File "/home/faten/anaconda3/envs/X/lib/python3.8/site-packages/ray/util/iter.py", line 783, in apply_foreach
    for item in it:
  File "/home/faten/anaconda3/envs/X/lib/python3.8/site-packages/ray/util/iter.py", line 783, in apply_foreach
    for item in it:
  [Previous line repeated 1 more time]
  File "/home/faten/anaconda3/envs/X/lib/python3.8/site-packages/ray/util/iter.py", line 471, in base_iterator
    yield ray.get(futures, timeout=timeout)
ray.exceptions.RayTaskError(AttributeError): ray::RolloutWorker.par_iter_next()
  File "python/ray/_raylet.pyx", line 463, in ray._raylet.execute_task
  File "python/ray/_raylet.pyx", line 415, in ray._raylet.execute_task.function_executor
  File "/home/faten/anaconda3/envs/X/lib/python3.8/site-packages/ray/util/iter.py", line 1152, in par_iter_next
    return next(self.local_it)
  File "/home/faten/anaconda3/envs/X/lib/python3.8/site-packages/ray/rllib/evaluation/rollout_worker.py", line 323, in gen_rollouts
    yield self.sample()
  File "/home/faten/anaconda3/envs/X/lib/python3.8/site-packages/ray/rllib/evaluation/rollout_worker.py", line 638, in sample
    batches = [self.input_reader.next()]
  File "/home/faten/anaconda3/envs/X/lib/python3.8/site-packages/ray/rllib/evaluation/sampler.py", line 94, in next
    batches = [self.get_data()]
  File "/home/faten/anaconda3/envs/X/lib/python3.8/site-packages/ray/rllib/evaluation/sampler.py", line 211, in get_data
    item = next(self.rollout_provider)
  File "/home/faten/anaconda3/envs/X/lib/python3.8/site-packages/ray/rllib/evaluation/sampler.py", line 568, in _env_runner
    _process_observations_w_trajectory_view_api(
  File "/home/faten/anaconda3/envs/X/lib/python3.8/site-packages/ray/rllib/evaluation/sampler.py", line 1070, in _process_observations_w_trajectory_view_api
    _sample_collector.add_action_reward_next_obs(
  File "/home/faten/anaconda3/envs/X/lib/python3.8/site-packages/ray/rllib/evaluation/collectors/simple_list_collector.py", line 420, in add_action_reward_next_obs
    self.agent_collectors[agent_key].add_action_reward_next_obs(values)
  File "/home/faten/anaconda3/envs/X/lib/python3.8/site-packages/ray/rllib/evaluation/collectors/simple_list_collector.py", line 106, in add_action_reward_next_obs
    self._build_buffers(single_row=values)
  File "/home/faten/anaconda3/envs/X/lib/python3.8/site-packages/ray/rllib/evaluation/collectors/simple_list_collector.py", line 209, in _build_buffers
    shape = data.shape
AttributeError: 'AgentId' object has no attribute 'shape'

When debugging I realized that the issue that happens here is due to the fact that it is expecting the data to be one of the types mentioned in the comments, but not an object.

In fact, in our multi-agent environment, the value of the key agent_id is an object called AgentId and doesn’t seem to be supported from my understanding.

Any pointers of what could be done at this point?
Thanks

2

Hi @faten,

Good luck! I had to do a similar migration and there are a lot or big architecture differences between those versions.

I think the intention is that the agent_id is a key that maps something in the environment to a policy that generates actions for it. Can you make the agent_id the str or repr of the AgentId object or will that break a lot of things in your set up ?

2 Likes
3

Hi @faten , this is actually a bug. This (still open) PR should fix it (for a related reason of being able to specify any type as data to be collected by the SimpleListCollector).
But then you wouldn’t be on 1.1 anymore :confused:
Maybe you can patch it? @mannyv is right, agent IDs are usually strings, but then again, we probably shouldn’t have this restriction.

1 Like