# Setting up multiagent config dict with different algorithm parameters

**URL:** <https://discuss.ray.io/t/setting-up-multiagent-config-dict-with-different-algorithm-parameters/8616>\
**Category:** RLlib\
**Created:** [December 10, 2022, 2:05pm UTC](https://discuss.ray.io/t/setting-up-multiagent-config-dict-with-different-algorithm-parameters/8616 "2022-12-10T14:05:03Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Blubberblub](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/blubberblub/32/1396_2.png) [@Blubberblub](https://discuss.ray.io/u/Blubberblub)\
**Post date:** [December 10, 2022, 2:05pm UTC](https://discuss.ray.io/t/setting-up-multiagent-config-dict-with-different-algorithm-parameters/8616/1 "2022-12-10T14:05:03Z")

</div>

**How severe does this issue affect your experience of using Ray?**

- Medium: It contributes to significant difficulty to complete my task, but I can work around it.

I want to specify different algorithm specific values (gamma and vf\_clip\_param) for my policies (both using PPO) since they should have different reward shaping and the height of the rewards is very different. However the algorithm only seems to change the values when i’m changing them in the config root. When changed in the multiagent config dict they are not updated. How am i supposed to set those up individually?

My config:

```auto
train_config = {
    "env": MyEnv,
    "env_config": env_config,
    "entropy_coeff": 0.01,
    # "vf_clip_param": 12,
    "multiagent": {
        "policies": {
            "pol_1": (
                None,
                Box(
                    low=0,
                    high=255,
                    shape=(41, 41, 2), # shape=(HEIGHT, WIDTH, N_CHANNELS)
                    dtype=np.uint8,
                ),
                Discrete(2),
                { 
                    "gamma": 0.95,
                    "vf_clip_param": 64,
                    "model": {
                        "no_final_linear": True,
                        "conv_filters": filters,
                    },
                },
            ),
            "pol_2": (
                None,
                Box(
                    low=0,
                    high=255,
                    shape=(41, 41, 2), # shape=(HEIGHT, WIDTH, N_CHANNELS)
                    dtype=np.uint8,
                ),
                Discrete(5),
                {
                    "gamma": 0.98,
                    "vf_clip_param": 20,
                    "model": {
                        "no_final_linear": True,
                        "conv_filters": filters
                    },
                },
            ),
        },
        "policy_mapping_fn": policy_mapping_fn,
        # Optional list of policies to train, or None for all policies.
        "policies_to_train": None,
    },
    "num_workers": 1,
    "framework": "torch",
    "log_level": "DEBUG",
    "num_cpus_for_driver": 2,
    "num_cpus_per_worker": 2,
    "num_gpus": 0.5,
    "num_gpus_per_worker": 0.5,
    "disable_env_checking": True,
    "train_batch_size": 4000
}

```

---

<div class="post-metadata">

**Author:** ![arturn](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/arturn/32/2096_2.png) [@arturn](https://discuss.ray.io/u/arturn)\
**Post date:** [December 15, 2022, 9:02am UTC](https://discuss.ray.io/t/setting-up-multiagent-config-dict-with-different-algorithm-parameters/8616/2 "2022-12-15T09:02:25Z")

</div>

Hi @Blubberblub ,

Before I look into this, could you try this with Ray 2.2?

Cheers

---

<div class="post-metadata">

**Author:** ![Blubberblub](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/blubberblub/32/1396_2.png) [@Blubberblub](https://discuss.ray.io/u/Blubberblub)\
**Post date:** [December 16, 2022, 11:06am UTC](https://discuss.ray.io/t/setting-up-multiagent-config-dict-with-different-algorithm-parameters/8616/3 "2022-12-16T11:06:09Z")

</div>

@arturn Yes, i will upgrade next week and check again.