# Variable-Sharing between policies

**URL:** <https://discuss.ray.io/t/variable-sharing-between-policies/1145>\
**Category:** RLlib\
**Created:** [March 5, 2021, 6:12pm UTC](https://discuss.ray.io/t/variable-sharing-between-policies/1145 "2021-03-05T18:12:12Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![klausk55](https://avatars.discourse-cdn.com/v4/letter/k/f17d59/32.png) [@klausk55](https://discuss.ray.io/u/klausk55)\
**Post date:** [March 5, 2021, 6:12pm UTC](https://discuss.ray.io/t/variable-sharing-between-policies/1145/1 "2021-03-05T18:12:13Z")

</div>

Hello,

I’m trying to share layers of my custom ModelV2 NN model between two policies.  
I tried as noted in the documentation [https://docs.ray.io/en/master/rllib-env.html#variable-sharing-between-policies](https://docs.ray.io/en/master/rllib-env.html#variable-sharing-between-policies) where is said I can just put layers in global variables and directly share those layer objects between policy models.

But already for the “first connection” between a local layer and a globally shared layer I get this error:  
`ValueError: Tensor("hoist1/dense_self_hoist/kernel/Read/ReadVariableOp:0", shape=(512, 256), dtype=float32) must be from the same graph as Tensor("hoist1/embedded_self_hoist/Relu:0", shape=(?, ?, 1, 512), dtype=float32) (graphs are <tensorflow.python.framework.ops.Graph object at 0x7f1e3033d9d0> and <tensorflow.python.framework.ops.Graph object at 0x7f1d2ef7f0a0>).`

What am I doing wrong? I globally share some layers in the manner showed in the example:  
[https://github.com/ray-project/ray/blob/ef944bc5f0d7764cd99d50500e470eac005a3d01/rllib/examples/models/shared\_weights\_model.py#L20](https://github.com/ray-project/ray/blob/ef944bc5f0d7764cd99d50500e470eac005a3d01/rllib/examples/models/shared_weights_model.py#L20)

---

<div class="post-metadata">

**Author:** ![klausk55](https://avatars.discourse-cdn.com/v4/letter/k/f17d59/32.png) [@klausk55](https://discuss.ray.io/u/klausk55)\
**Post date:** [March 8, 2021, 12:46pm UTC](https://discuss.ray.io/t/variable-sharing-between-policies/1145/2 "2021-03-08T12:46:20Z")

</div>

I forgot to set `framework='tf2'` which seems to be mandatory for sharing layers as global variables between models.  
But thereafter it occurs another value error:

> ValueError: tf.enable\_eager\_execution must be called at program startup

I have opened an issue, see [ValueError: tf.enable\_eager\_execution must be called at program startup · Issue #14533 · ray-project/ray · GitHub](https://github.com/ray-project/ray/issues/14533#issue-824485336)

---

<div class="post-metadata">

**Author:** ![sven1977](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/sven1977/32/53_2.png) [@sven1977](https://discuss.ray.io/u/sven1977)\
**Post date:** [March 12, 2021, 8:12pm UTC](https://discuss.ray.io/t/variable-sharing-between-policies/1145/3 "2021-03-12T20:12:43Z")

</div>

Thanks for this issue @klausk55 and for the repro script! Yeah, sharing layers like this is only possible in torch or tf-eager. In static-graph tf, you would have to do this via (old-school) tf scopes (and var re-uses).  
I’ll take a look …

---

<div class="post-metadata">

**Author:** ![klausk55](https://avatars.discourse-cdn.com/v4/letter/k/f17d59/32.png) [@klausk55](https://discuss.ray.io/u/klausk55)\
**Post date:** [March 13, 2021, 7:45pm UTC](https://discuss.ray.io/t/variable-sharing-between-policies/1145/4 "2021-03-13T19:45:25Z")

</div>

Okay @sven1977, do you will take a look at why it’s working using the default (`results = tune.run("PPO", stop=stop, config=config, verbose=1`) in that example?

As you said, so far I have to use the old-school manner to run the `PPOTrainer` w/o including `tune`.