# RLlib: vf\_share\_layers defined in multiple places

**URL:** <https://discuss.ray.io/t/rllib-vf-share-layers-defined-in-multiple-places/421>\
**Category:** RLlib\
**Created:** [January 11, 2021, 11:53pm UTC](https://discuss.ray.io/t/rllib-vf-share-layers-defined-in-multiple-places/421 "2021-01-11T23:53:10Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![andrew-rosenfeld-ts](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/andrew-rosenfeld-ts/32/37_2.png) [@andrew-rosenfeld-ts](https://discuss.ray.io/u/andrew-rosenfeld-ts)\
**Post date:** [January 11, 2021, 11:53pm UTC](https://discuss.ray.io/t/rllib-vf-share-layers-defined-in-multiple-places/421/1 "2021-01-11T23:53:10Z")

</div>

I’ve noticed that several of the RLlib [algorithms](https://docs.ray.io/en/latest/rllib-algorithms.html) have a top-level config parameter named `vf_share_layers`.

However, `vf_share_layers` is also present in the nested [model config](https://docs.ray.io/en/latest/rllib-models.html).

Is this a bug? Is this intentional? Which to use?

---

<div class="post-metadata">

**Author:** ![sven1977](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/sven1977/32/53_2.png) [@sven1977](https://discuss.ray.io/u/sven1977)\
**Post date:** [January 13, 2021, 9:54am UTC](https://discuss.ray.io/t/rllib-vf-share-layers-defined-in-multiple-places/421/2 "2021-01-13T09:54:36Z")

</div>

I agree, this is very confusing! Thanks for raising awareness for this.

There is a function in ppo\_tf\_policy.py, which overrides the model config with the top-level one, so the answer is to always use the top-level config key: `vf_share_layers` to set this, if available:

```auto
def setup_config(...):
   ...
   # Auto set the model option for VF layer sharing.
   config["model"]["vf_share_layers"] = config["vf_share_layers"]

```

I’ll fix this inconsistency and soft-deprecate the top-level config key. That way, if users correctly set the model’s config `vf_share_layers`, they don’t get bad surprises (b/c it’s silently overwritten by the Trainer’s value).

---

<div class="post-metadata">

**Author:** ![sven1977](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/sven1977/32/53_2.png) [@sven1977](https://discuss.ray.io/u/sven1977)\
**Post date:** [January 13, 2021, 9:56am UTC](https://discuss.ray.io/t/rllib-vf-share-layers-defined-in-multiple-places/421/3 "2021-01-13T09:56:43Z")

</div>

So the workaround for now is:  
Only use `vf_share_layers` in the top-level config for PPO, MAML, and MB-MPO, but not for any other algo.  
For PPO, MAML, and MB-MPO, you **must** use the top-level key as the model config key will always be silently overwritten.

---

<div class="post-metadata">

**Author:** ![sven1977](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/sven1977/32/53_2.png) [@sven1977](https://discuss.ray.io/u/sven1977)\
**Post date:** [January 13, 2021, 10:00am UTC](https://discuss.ray.io/t/rllib-vf-share-layers-defined-in-multiple-places/421/4 "2021-01-13T10:00:34Z")

</div>

> <https://github.com/ray-project/ray/issues/13396>
>
> Andrew R. raised this on discuss.ray.io:
> I’ve noticed that several of the RLlib algorithms have a top-level config parameter named vf\_share\_layers.
> However, vf\_share\_layers...