# Empty action\_dict coming from policy net

**URL:** https://discuss.ray.io/t/empty-action-dict-coming-from-policy-net/10462
**Category:** RLlib
**Created:** [May 1, 2023, 4:58pm UTC](https://discuss.ray.io/t/empty-action-dict-coming-from-policy-net/10462 "2023-05-01T16:58:41Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![Username1](https://avatars.discourse-cdn.com/v4/letter/u/ee59a6/32.png) [@Username1](https://discuss.ray.io/u/Username1)
#### Post date: [May 1, 2023, 4:58pm UTC](https://discuss.ray.io/t/empty-action-dict-coming-from-policy-net/10462/1 "2023-05-01T16:58:41Z")

</div>

**How severe does this issue affect your experience of using Ray?**

- Medium: It contributes to significant difficulty to complete my task, but I can work around it.

I am using Ray 2.2.0 in a multi-agent custom environment. After the reset method, the action dictionary coming from the policy network is empty.

I am leaving a working example below. It is a very simple environment trained with Tune and Air. I might be doing something wrong but don’t know where.

I have added a print statement on the environment to print out whether the action\_dict is empty. As follows:

```auto
    def step(self, action_dict):
        
        self.t +=1
        
        if not action_dict:
           print("EMPTY ACTION DICT!!!")
           print('self.t =', self.t)

```

The code can be found here:

> <https://github.com/lcipolina/Ray_tutorials/blob/main/RLLIB_MARL_Empty_action_Ray_2_2_0.ipynb>

> **[Google Colaboratory](https://colab.research.google.com/drive/1tegQZRig1qntJ77r3ENffckEssZa0zx9#scrollTo=vG62SSDSK50i)**

---

<div class="post-metadata">

### Author: ![Rohan138](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/rohan138/32/1160_2.png) [@Rohan138](https://discuss.ray.io/u/Rohan138)
#### Post date: [May 22, 2023, 11:35pm UTC](https://discuss.ray.io/t/empty-action-dict-coming-from-policy-net/10462/2 "2023-05-22T23:35:40Z")

</div>

from [ray/multi\_agent\_env.py at master · ray-project/ray · GitHub](https://github.com/ray-project/ray/blob/master/rllib/env/multi_agent_env.py#L37:)

```auto
  The preferred format for action- and observation space is a mapping from agent
  ids to their individual spaces. If that is not provided, the respective methods'
  observation_space_contains(), action_space_contains(),
  action_space_sample() and observation_space_sample() have to be overwritten.

```

In your example, you should rewrite the obs and action spaces to be `Dict()` or modify the methods listed above.

---

<div class="post-metadata">

### Author: ![Username1](https://avatars.discourse-cdn.com/v4/letter/u/ee59a6/32.png) [@Username1](https://discuss.ray.io/u/Username1)
#### Post date: [May 26, 2023, 12:16pm UTC](https://discuss.ray.io/t/empty-action-dict-coming-from-policy-net/10462/3 "2023-05-26T12:16:55Z")

</div>

> [@Rohan138](#):
>
> `observation_space_contains()`

Thank you @Rohan138, I can’t work with dictionaries because it makes no sense for my environment. So I’ll have to go with Tuples.

How do I implement the methods mentioned:

```auto
observation_space_contains(), action_space_contains(),
  action_space_sample() and observation_space_sample() 

```

Is there any example of how they should be set up?

Thanks!