# Searching Across Environment Configurations

**URL:** <https://discuss.ray.io/t/searching-across-environment-configurations/6382>\
**Category:** RLlib\
**Created:** [June 3, 2022, 3:43pm UTC](https://discuss.ray.io/t/searching-across-environment-configurations/6382 "2022-06-03T15:43:11Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Shaun\_Fattig](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/shaun_fattig/32/2670_2.png) [@Shaun\_Fattig](https://discuss.ray.io/u/Shaun_Fattig)\
**Post date:** [June 3, 2022, 3:43pm UTC](https://discuss.ray.io/t/searching-across-environment-configurations/6382/1 "2022-06-03T15:43:11Z")

</div>

- Medium: It contributes to significant difficulty to complete my task, but I can work around it.

What is the best way to search different environment configurations using Tune?  
I found the `env_config` key, val that can be passed in with the `config` option of `tune.run()`, where a `tune.<search>` algorithm can be used to search across various settings. However, I’m training a multi-agent environment where the policies need to have the observation and action space given to it before `tune.run()` is executed. Currently I’m achieving this by creating a sample environment just before `tune.run()` and extracting the `sample_env.observation_space` and `sample_env.action_space`.  
Unless some input validation or something is happening, I’m assuming the policies only need to know about the shape of the observations/actions. However, if the shape changes with each environment configuration, there is a mismatch between policy and environment and an error is thrown.

What is the proper way to handle this situation?

---

<div class="post-metadata">

**Author:** ![amogkam](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/amogkam/32/17_2.png) [@amogkam](https://discuss.ray.io/u/amogkam)\
**Post date:** [June 3, 2022, 8:50pm UTC](https://discuss.ray.io/t/searching-across-environment-configurations/6382/2 "2022-06-03T20:50:29Z")

</div>

Hey @Shaun_Fattig! Just to clarify are you using RLlib here?

---

<div class="post-metadata">

**Author:** ![Shaun\_Fattig](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/shaun_fattig/32/2670_2.png) [@Shaun\_Fattig](https://discuss.ray.io/u/Shaun_Fattig)\
**Post date:** [June 6, 2022, 2:25pm UTC](https://discuss.ray.io/t/searching-across-environment-configurations/6382/3 "2022-06-06T14:25:02Z")

</div>

@amogkam Correct, using RLlib

---

<div class="post-metadata">

**Author:** ![amogkam](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/amogkam/32/17_2.png) [@amogkam](https://discuss.ray.io/u/amogkam)\
**Post date:** [June 6, 2022, 9:22pm UTC](https://discuss.ray.io/t/searching-across-environment-configurations/6382/4 "2022-06-06T21:22:55Z")

</div>

Got it thanks. cc @gjoliver could take a look?

---

<div class="post-metadata">

**Author:** ![gjoliver](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/gjoliver/32/1490_2.png) [@gjoliver](https://discuss.ray.io/u/gjoliver)\
**Post date:** [June 6, 2022, 11:31pm UTC](https://discuss.ray.io/t/searching-across-environment-configurations/6382/5 "2022-06-06T23:31:52Z")

</div>

sorry I don’t get why you need to create your env before Tune runs.  
can you register an env creation function for your env that takes in env\_cfg and creates an corresponding env?  
E.g.,

> **[rock\_paper\_scissors\_multiagent.py?L69:1](https://sourcegraph.com/github.com/ray-project/ray/-/blob/rllib/examples/rock_paper_scissors_multiagent.py?L69:1)**
>
> https://sourcegraph.com/github.com/ray-project/ray

---

<div class="post-metadata">

**Author:** ![Shaun\_Fattig](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/shaun_fattig/32/2670_2.png) [@Shaun\_Fattig](https://discuss.ray.io/u/Shaun_Fattig)\
**Post date:** [June 7, 2022, 6:42pm UTC](https://discuss.ray.io/t/searching-across-environment-configurations/6382/6 "2022-06-07T18:42:26Z")

</div>

@gjoliver

I’m passing the policies into the ‘multiagent’ key of the Tune config parameter, and each policy requires observations and actions to be specified beforehand.

> <https://github.com/ray-project/ray/blob/ray-1.6.0/rllib/agents/trainer.py#L430-L463>

```auto
ma_config = {
    ...,
    policies = {
        'policy1': (policy_cls1, obs_space1, act_space1, config1),
        'policy2': (policy_cls2, obs_space2, act_space2, config2)
    }
}

tune.run(
    ...,
    config = {
        ...,
        'multiagent': ma_config
    }
)

```

I see in that example environment that `PolicySpec` is being used to initialize the policies. I’m not familiar with this class, so that may hold keys to my issues.

By the way, I’m currently using Ray 1.6

---

<div class="post-metadata">

**Author:** ![gjoliver](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/gjoliver/32/1490_2.png) [@gjoliver](https://discuss.ray.io/u/gjoliver)\
**Post date:** [June 8, 2022, 8:47am UTC](https://discuss.ray.io/t/searching-across-environment-configurations/6382/7 "2022-06-08T08:47:18Z")

</div>

Any reason you are using a 6 month old version?

I think you can just specify None for the spaces, which RLlib would simply use the Env to determine space types:

> <https://github.com/ray-project/ray/blob/ray-1.6.0/rllib/policy/policy.py#L47-L52>

Hope this helps.