# Policy mapping and agentIDs in hierachical env example

**URL:** <https://discuss.ray.io/t/policy-mapping-and-agentids-in-hierachical-env-example/3241>\
**Category:** RLlib\
**Created:** [August 13, 2021, 9:53am UTC](https://discuss.ray.io/t/policy-mapping-and-agentids-in-hierachical-env-example/3241 "2021-08-13T09:53:54Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Blubberblub](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/blubberblub/32/1396_2.png) [@Blubberblub](https://discuss.ray.io/u/Blubberblub)\
**Post date:** [August 13, 2021, 9:53am UTC](https://discuss.ray.io/t/policy-mapping-and-agentids-in-hierachical-env-example/3241/1 "2021-08-13T09:53:55Z")

</div>

Hey everyone,

i’m writing a custom multi-agent environment that should implement multiple hierarchical agents.

So i looked into the hierarchical\_training.py example that uses the HierarchicalWindyMazeEnv. However i’m getting confused how to exactly implement the correct agents IDs in the environment code.

In the policy mapping function from hierarchical\_training.py the policies are mapped by agent ids. But in the HierarchicalWindyMazeEnv the agent ids are implemented in two ways:

1. as key for the observation return in the reset function in case of the high\_level\_agent  
and
2. with self.low\_level\_agent\_id for the low\_level\_agent.

Opposed to this the FlexAgentsMultiAgent example class from the multi\_agent.py example sets the agent ids while initializing the agents to an self.agents dict (using int values as opposed to strings).

What’s the best practice here and how exactly does tune or the trainer connects the policies to the agent ids while running?

File references:

> <https://github.com/ray-project/ray/blob/master/rllib/examples/hierarchical_training.py>

> <https://github.com/ray-project/ray/blob/master/rllib/examples/env/windy_maze_env.py>

> <https://github.com/ray-project/ray/blob/master/rllib/examples/env/multi_agent.py>