# Alert \`Policy\_Client\`s when \`Policy\_Server\` completes an epoch

**URL:** <https://discuss.ray.io/t/alert-policy-client-s-when-policy-server-completes-an-epoch/9936>\
**Category:** RLlib\
**Created:** [March 28, 2023, 7:01pm UTC](https://discuss.ray.io/t/alert-policy-client-s-when-policy-server-completes-an-epoch/9936 "2023-03-28T19:01:26Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Denys\_Ashikhin](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/denys_ashikhin/32/58_2.png) [@Denys\_Ashikhin](https://discuss.ray.io/u/Denys_Ashikhin)\
**Post date:** [March 28, 2023, 7:01pm UTC](https://discuss.ray.io/t/alert-policy-client-s-when-policy-server-completes-an-epoch/9936/1 "2023-03-28T19:01:26Z")

</div>

**How severe does this issue affect your experience of using Ray?**

- Medium: It contributes to significant difficulty to complete my task, but I can work around it.

Hi all,

I’m using the policy\_server + policy\_client to do some training on remote clients. My question is if there is any way for a client to know if a policy\_server either: started, is in the process of, or completed an epoch?

The reason being, the training epoch takes ~25 seconds. During this time, the clients are stuck waiting for new actions and that gap time causes problems that I could handle more gracefully. (Currently I’m using times between received actions to scrap previous environments that may have completed in that ~25 seconds).

So if there is any way, please let me know - or the closest workaround, thanks!

---

<div class="post-metadata">

**Author:** ![Lars\_Simon\_Zehnder](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/lars_simon_zehnder/32/1185_2.png) [@Lars\_Simon\_Zehnder](https://discuss.ray.io/u/Lars_Simon_Zehnder)\
**Post date:** [March 30, 2023, 11:37am UTC](https://discuss.ray.io/t/alert-policy-client-s-when-policy-server-completes-an-epoch/9936/2 "2023-03-30T11:37:55Z")

</div>

Hi @Denys_Ashikhin ,

you might want to take a look into the [`Handler` class](https://github.com/ray-project/ray/blob/fbaba01efbe0709e12fd6b51ec1a40553feb232a/rllib/env/policy_server_input.py#L229) of the `PolicyServerInput` and override the communication (sending to clients also some information about training).

---

<div class="post-metadata">

**Author:** ![Denys\_Ashikhin](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/denys_ashikhin/32/58_2.png) [@Denys\_Ashikhin](https://discuss.ray.io/u/Denys_Ashikhin)\
**Post date:** [March 30, 2023, 3:39pm UTC](https://discuss.ray.io/t/alert-policy-client-s-when-policy-server-completes-an-epoch/9936/3 "2023-03-30T15:39:11Z")

</div>

If I understood correctly: [ray/policy\_server\_input.py at fbaba01efbe0709e12fd6b51ec1a40553feb232a · ray-project/ray · GitHub](https://github.com/ray-project/ray/blob/fbaba01efbe0709e12fd6b51ec1a40553feb232a/rllib/env/policy_server_input.py#L272)

I can just modify it to:

```auto
elif command == Commands.GET_ACTION:
                assert inference_thread.is_alive()
                response["action"] = child_rollout_worker.env.get_action(
                    args["episode_id"], args["observation"], {"custom object here": 1234}
                )

```

And then on policy\_client:

```auto
action, customObject = client.get_action(episode_id=episode_id, observation=obs)

```

And then I get my customObject which would have whatever it is I need?