# Purpose and working of the vf\_share\_layers parameter

**URL:** <https://discuss.ray.io/t/purpose-and-working-of-the-vf-share-layers-parameter/3253>\
**Category:** RLlib\
**Created:** [August 14, 2021, 2:39pm UTC](https://discuss.ray.io/t/purpose-and-working-of-the-vf-share-layers-parameter/3253 "2021-08-14T14:39:03Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Rohan\_James](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/rohan_james/32/971_2.png) [@Rohan\_James](https://discuss.ray.io/u/Rohan_James)\
**Post date:** [August 14, 2021, 2:39pm UTC](https://discuss.ray.io/t/purpose-and-working-of-the-vf-share-layers-parameter/3253/1 "2021-08-14T14:39:03Z")

</div>

Hi all! This is a question I posted on slack recently, which @sven1977 answered. Posting it here for broader reach.

I’ve been trying to understand how the config parameter `vf_share_layers` affects learning, and I had a few questions. Would be grateful if someone could throw some light on any of these!

- While **implementing a custom model** , does toggling the value of `vf_share_layers` change learning behavior? If so, how? Asking because a [Github search](https://github.com/ray-project/ray/search?l=Python&p=1&q=vf_share_layers) of the parameter showed me the parameter was used only in the _existing models_ inside Rllib.
- When vf losses are high, how does disabling `vf_share_layers` [alleviate the issue](https://github.com/ray-project/ray/blob/5a313ba3d6b086c6613ff2f5c5d419bcac0f1921/rllib/agents/ppo/ppo.py#L221)?
- And why does `vf_loss_coeff` [need to be tuned](https://github.com/ray-project/ray/blob/5a313ba3d6b086c6613ff2f5c5d419bcac0f1921/rllib/agents/ppo/ppo.py#L66) when `vf_share_layers` is true?

[Here](https://ray-distributed.slack.com/archives/CMVUQ22JD/p1628886548018500?thread_ts=1628778990.014400&cid=CMVUQ22JD) is the link to Sven’s answer, thanks again!

---

<div class="post-metadata">

**Author:** ![sven1977](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/sven1977/32/53_2.png) [@sven1977](https://discuss.ray.io/u/sven1977)\
**Post date:** [August 18, 2021, 8:05am UTC](https://discuss.ray.io/t/purpose-and-working-of-the-vf-share-layers-parameter/3253/2 "2021-08-18T08:05:35Z")

</div>

Great question and yes, vf\_share\_layers is indeed a source for confusion. Yes, it’s only useful for non-custom (default) models, unless your custom model reads and respects this parameter, of course. RLlib’s default models (fcnet and visionnet) have the functionality to build both: a) a core net + policy-head + value-head or b) a policy-net and an independent value-net, depending on that parameter.  
I think if vf\_losses are too high, you should first try to set the vf\_loss\_coeff lower, but yes, even playing with vf\_share\_layers (when using default models) may help. There is also the option of making the stddev output nodes (for cont. actions) completely independent, learnable bias values that are outside the other action output: free\_log\_std:

```auto
    # For DiagGaussian action distributions, make the second half of the model
    # outputs floating bias variables instead of state-dependent. This only
    # has an effect is using the default fully connected net.
    "free_log_std": False,

```