# Hierachical multi-agent RL

**URL:** <https://discuss.ray.io/t/hierachical-multi-agent-rl/9382>\
**Category:** RLlib\
**Created:** [February 15, 2023, 11:01am UTC](https://discuss.ray.io/t/hierachical-multi-agent-rl/9382 "2023-02-15T11:01:30Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![oximi123](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/oximi123/32/3899_2.png) [@oximi123](https://discuss.ray.io/u/oximi123)\
**Post date:** [February 15, 2023, 11:01am UTC](https://discuss.ray.io/t/hierachical-multi-agent-rl/9382/1 "2023-02-15T11:01:30Z")

</div>

Hey guys. Assume I have a two-level HRL algorithm, where a upper-level agent decides which sub-task to perform (two subtasks). For each task, I want to use multi-agent RL to solve it. That means each subtask have its own multi-agents to solve it. So the RL workflow is, at each time stamp, the upper level agent frist choose a subtasks. Then, the agents of the chosen tasks act simutaneously for serveral steps to solve that sub-task. When the subtask is solved, the upper-level agent will get the reward.  
Can anyone give me some advice on how to implement my idea.

---

<div class="post-metadata">

**Author:** ![mannyv](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/mannyv/32/606_2.png) [@mannyv](https://discuss.ray.io/u/mannyv)\
**Post date:** [February 15, 2023, 11:35am UTC](https://discuss.ray.io/t/hierachical-multi-agent-rl/9382/2 "2023-02-15T11:35:40Z")

</div>

Hi @oximi123,

Welcome to the forums.

I think you could adapt this example. Maybe inside your environment you keep two sub-environments and switch between them based on the action of the higher environment.

> <https://github.com/ray-project/ray/blob/master/rllib/examples/hierarchical_training.py>