# Independent learning for more agents \[PettingZoo waterworld\_v4\]

**URL:** <https://discuss.ray.io/t/independent-learning-for-more-agents-pettingzoo-waterworld-v4/21347>\
**Category:** Configure Algorithm, Training, Evaluation, Scaling\
**Created:** [January 2, 2025, 4:14pm UTC](https://discuss.ray.io/t/independent-learning-for-more-agents-pettingzoo-waterworld-v4/21347 "2025-01-02T16:14:08Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![panayiotav](https://avatars.discourse-cdn.com/v4/letter/p/cab0a1/32.png) [@panayiotav](https://discuss.ray.io/u/panayiotav)\
**Post date:** [January 2, 2025, 4:14pm UTC](https://discuss.ray.io/t/independent-learning-for-more-agents-pettingzoo-waterworld-v4/21347/1 "2025-01-02T16:14:08Z")

</div>

Hi,

I am new to this library, I would like to have multiple independent agents learning in the same environment.

The waterworld example [here](https://github.com/ray-project/ray/blob/master/rllib/examples/multi_agent/pettingzoo_independent_learning.py) is close to what I want to achieve but only works for two agents (pursuers). How can I adjust it to work for 5 agents for example?

This is the error that I get when I increase the number of agents to 5, i.e. set n\_pursuers = 5 and args.num\_agents = 5:

```auto
(MultiAgentEnvRunner pid=140923) 2025-01-02 18:07:36,256 ERROR actor_manager.py:187 -- Worker exception caught during `apply()`: all input arrays must have the same shape
(MultiAgentEnvRunner pid=140923) File "/home/pvalia01/miniconda3/envs/pettingzooenv2/lib/python3.11/site-packages/ray/rllib/utils/actor_manager.py", line 183, in apply
(MultiAgentEnvRunner pid=140923) return func(self, *args, **kwargs)
(MultiAgentEnvRunner pid=140923) ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(MultiAgentEnvRunner pid=140923) File "/home/pvalia01/miniconda3/envs/pettingzooenv2/lib/python3.11/site-packages/ray/rllib/execution/rollout_ops.py", line 110, in <lambda>
(MultiAgentEnvRunner pid=140923) else (lambda w: (w.sample(**random_action_kwargs), w.get_metrics()))
(MultiAgentEnvRunner pid=140923) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(MultiAgentEnvRunner pid=140923) File "/home/pvalia01/miniconda3/envs/pettingzooenv2/lib/python3.11/site-packages/ray/util/tracing/tracing_helper.py", line 467, in _resume_span
(MultiAgentEnvRunner pid=140923) return method(self, *_args, **_kwargs)
(MultiAgentEnvRunner pid=140923) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(MultiAgentEnvRunner pid=140923) File "/home/pvalia01/miniconda3/envs/pettingzooenv2/lib/python3.11/site-packages/ray/rllib/env/multi_agent_env_runner.py", line 180, in sample
(MultiAgentEnvRunner pid=140923) samples = self._sample_timesteps(
(MultiAgentEnvRunner pid=140923) ^^^^^^^^^^^^^^^^^^^^^^^
(MultiAgentEnvRunner pid=140923) File "/home/pvalia01/miniconda3/envs/pettingzooenv2/lib/python3.11/site-packages/ray/util/tracing/tracing_helper.py", line 467, in _resume_span
(MultiAgentEnvRunner pid=140923) return method(self, *_args, **_kwargs)
(MultiAgentEnvRunner pid=140923) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(MultiAgentEnvRunner pid=140923) File "/home/pvalia01/miniconda3/envs/pettingzooenv2/lib/python3.11/site-packages/ray/rllib/env/multi_agent_env_runner.py", line 380, in _sample_timesteps
(MultiAgentEnvRunner pid=140923) self._episode.finalize(drop_zero_len_single_agent_episodes=True)
(MultiAgentEnvRunner pid=140923) File "/home/pvalia01/miniconda3/envs/pettingzooenv2/lib/python3.11/site-packages/ray/rllib/env/multi_agent_episode.py", line 794, in finalize
(MultiAgentEnvRunner pid=140923) agent_eps.finalize()
(MultiAgentEnvRunner pid=140923) File "/home/pvalia01/miniconda3/envs/pettingzooenv2/lib/python3.11/site-packages/ray/rllib/env/single_agent_episode.py", line 578, in finalize
(MultiAgentEnvRunner pid=140923) self.actions.finalize()
(MultiAgentEnvRunner pid=140923) File "/home/pvalia01/miniconda3/envs/pettingzooenv2/lib/python3.11/site-packages/ray/rllib/env/utils/infinite_lookback_buffer.py", line 161, in finalize
(MultiAgentEnvRunner pid=140923) self.data = batch(self.data)
(MultiAgentEnvRunner pid=140923) ^^^^^^^^^^^^^^^^
(MultiAgentEnvRunner pid=140923) File "/home/pvalia01/miniconda3/envs/pettingzooenv2/lib/python3.11/site-packages/ray/rllib/utils/spaces/space_utils.py", line 373, in batch
(MultiAgentEnvRunner pid=140923) ret = tree.map_structure(lambda *s: np_func(s, axis=0), *list_of_structs)
(MultiAgentEnvRunner pid=140923) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(MultiAgentEnvRunner pid=140923) File "/home/pvalia01/miniconda3/envs/pettingzooenv2/lib/python3.11/site-packages/tree/ __init__.py", line 435, in map_structure
(MultiAgentEnvRunner pid=140923) [func(*args) for args in zip(*map(flatten, structures))])
(MultiAgentEnvRunner pid=140923) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(MultiAgentEnvRunner pid=140923) File "/home/pvalia01/miniconda3/envs/pettingzooenv2/lib/python3.11/site-packages/tree/ __init__.py", line 435, in <listcomp>
(MultiAgentEnvRunner pid=140923) [func(*args) for args in zip(*map(flatten, structures))])
(MultiAgentEnvRunner pid=140923) ^^^^^^^^^^^
(MultiAgentEnvRunner pid=140923) File "/home/pvalia01/miniconda3/envs/pettingzooenv2/lib/python3.11/site-packages/ray/rllib/utils/spaces/space_utils.py", line 373, in <lambda>
(MultiAgentEnvRunner pid=140923) ret = tree.map_structure(lambda *s: np_func(s, axis=0), *list_of_structs)
(MultiAgentEnvRunner pid=140923) ^^^^^^^^^^^^^^^^^^
(MultiAgentEnvRunner pid=140923) File "/home/pvalia01/miniconda3/envs/pettingzooenv2/lib/python3.11/site-packages/numpy/_core/shape_base.py", line 460, in stack
(MultiAgentEnvRunner pid=140923) raise ValueError('all input arrays must have the same shape')
(MultiAgentEnvRunner pid=140923) ValueError: all input arrays must have the same shape

```

Thank you.