# Is there a way to log the weight changes of neural networks?

**URL:** <https://discuss.ray.io/t/is-there-a-way-to-log-the-weight-changes-of-neural-networks/495>\
**Category:** RLlib\
**Created:** [January 20, 2021, 8:44am UTC](https://discuss.ray.io/t/is-there-a-way-to-log-the-weight-changes-of-neural-networks/495 "2021-01-20T08:44:39Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Kai\_Yun](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/kai_yun/32/167_2.png) [@Kai\_Yun](https://discuss.ray.io/u/Kai_Yun)\
**Post date:** [January 20, 2021, 8:44am UTC](https://discuss.ray.io/t/is-there-a-way-to-log-the-weight-changes-of-neural-networks/495/1 "2021-01-20T08:44:39Z")

</div>

Hello,

I would like to visualize how the weights are changing as a neural network continues its training. I’m prototyping this with RLlib DQN in simple Gym environments. [Weights and Biases](https://wandb.ai/) provides `watch()` method that logs the weight and gradients, but I’m having difficulty finding out how to integrate that into Ray Tune and RLlib.

Thanks in advance.

---

<div class="post-metadata">

**Author:** ![sven1977](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/sven1977/32/53_2.png) [@sven1977](https://discuss.ray.io/u/sven1977)\
**Post date:** [January 21, 2021, 8:42am UTC](https://discuss.ray.io/t/is-there-a-way-to-log-the-weight-changes-of-neural-networks/495/2 "2021-01-21T08:42:18Z")

</div>

Hey, you could add the weights inside the `on_train_result` callback (override the `rllib/agents/callbacks.py::DefaultCallbacks` class and implement `on_train_result` method). In there is a `result` arg, which is a dict to which you can add arbitrary data (e.g. your weights, which you can get in that method via `trainer.get_policy().get_weights()`).

---

<div class="post-metadata">

**Author:** ![Jakub\_Staniszewski](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/jakub_staniszewski/32/1610_2.png) [@Jakub\_Staniszewski](https://discuss.ray.io/u/Jakub_Staniszewski)\
**Post date:** [October 13, 2021, 7:16pm UTC](https://discuss.ray.io/t/is-there-a-way-to-log-the-weight-changes-of-neural-networks/495/3 "2021-10-13T19:16:48Z")

</div>

When I tried this I was able only to get the value with shape = 1 using W&B:

 ![image](https://us1.discourse-cdn.com/flex020/uploads/ray/original/2X/b/ba3ed75b7719d6dc02361df2d4e58fe8e0f5762b.png)

There is the way I tried to logged it:

```auto
class MyCallbacks(DefaultCallbacks):
    def on_train_result(self, trainer, result: dict, **kwargs):
        for k, v in trainer.get_policy().get_weights().items():
            result["FCC/{}".format(k)] = v

```

How can I log also other weights other than value\_out/bias?  
It seems that there is skipped all other arrays.  
I also tried with v.flatten() version.

---

<div class="post-metadata">

**Author:** ![bmanczak](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/bmanczak/32/1802_2.png) [@bmanczak](https://discuss.ray.io/u/bmanczak)\
**Post date:** [December 7, 2021, 12:06pm UTC](https://discuss.ray.io/t/is-there-a-way-to-log-the-weight-changes-of-neural-networks/495/4 "2021-12-07T12:06:32Z")

</div>

> [@Jakub\_Staniszewski](#):
>
> ```auto
> class MyCallbacks(DefaultCallbacks):
> def on_train_result(self, trainer, result: dict, **kwargs):
> for k, v in trainer.get_policy().get_weights().items():
> result["FCC/{}".format(k)] = v
> 
> ```

A bit of a tangent but worth a shot:

how did you make `MyCallbacks` compatible with W&B? Were you using tune or RLlib Trainer directly?

I am trying to make the logs from callbacks inheriting from `DefaultCallbacks` appear in W&B but can’t get that to work. I am using `tune.run` to train (as opposed to RLlib Trainer)

---

<div class="post-metadata">

**Author:** ![felipeeeantunes](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/felipeeeantunes/32/86_2.png) [@felipeeeantunes](https://discuss.ray.io/u/felipeeeantunes)\
**Post date:** [December 8, 2021, 2:57pm UTC](https://discuss.ray.io/t/is-there-a-way-to-log-the-weight-changes-of-neural-networks/495/5 "2021-12-08T14:57:25Z")

</div>

Sven I think this callback is so useful that deserves an example in the docs!