# How RLlib distinguish terminated/truncated situation in Server-client configuration?

**URL:** <https://discuss.ray.io/t/how-rllib-distinguish-terminated-truncated-situation-in-server-client-configuration/12097>\
**Category:** RLlib\
**Created:** [September 11, 2023, 8:09am UTC](https://discuss.ray.io/t/how-rllib-distinguish-terminated-truncated-situation-in-server-client-configuration/12097 "2023-09-11T08:09:09Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![tiankaidong](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/tiankaidong/32/5027_2.png) [@tiankaidong](https://discuss.ray.io/u/tiankaidong)\
**Post date:** [September 11, 2023, 8:09am UTC](https://discuss.ray.io/t/how-rllib-distinguish-terminated-truncated-situation-in-server-client-configuration/12097/1 "2023-09-11T08:09:09Z")

</div>

**How severe does this issue affect your experience of using Ray?**

- Medium: It contributes to significant difficulty to complete my task, but I can work around it.

I am following cartpole server-client example to build my task. in this client side, related key commands are as follows:  
client = PolicyClient()  
eid = client.start\_episode()  
for loop:  
action = client.get\_action(eid, obs)  
obs, reward, terminated, truncated, info = env.step(action)  
client.log\_returns(eid, reward, info=info)  
if terminated or truncated:  
client.end\_episode(eid, obs)  
It seems terminated/truncated values returned by env.step never passed to client api, how server side know if the episode end by truncated or terminated?  
I think the principle for terminated/truncated is quite different.  
for q learning type of algorithm, for terminated: q\_ground\_truth = reward, but for truncated: q\_ground\_truth = reward + q\_next\_step\_status (gamma ignored).  
is my understand about terminated/truncated right? How terminated/truncated is distinguished by server?

---

<div class="post-metadata">

**Author:** ![Lars\_Simon\_Zehnder](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/lars_simon_zehnder/32/1185_2.png) [@Lars\_Simon\_Zehnder](https://discuss.ray.io/u/Lars_Simon_Zehnder)\
**Post date:** [September 13, 2023, 8:28am UTC](https://discuss.ray.io/t/how-rllib-distinguish-terminated-truncated-situation-in-server-client-configuration/12097/2 "2023-09-13T08:28:53Z")

</div>

@tiankaidong great question! If you look into the [Unity3D example](https://github.com/ray-project/ray/blob/e44657390b375e703d1d164b2c26467d8e713e63/rllib/examples/serving/unity3d_client.py#L119) it becomes a bit clearer: we need to use the `multiagent_done_dict` inside the `log_returns()` function.

---

<div class="post-metadata">

**Author:** ![tiankaidong](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/tiankaidong/32/5027_2.png) [@tiankaidong](https://discuss.ray.io/u/tiankaidong)\
**Post date:** [September 14, 2023, 6:56am UTC](https://discuss.ray.io/t/how-rllib-distinguish-terminated-truncated-situation-in-server-client-configuration/12097/3 "2023-09-14T06:56:24Z")

</div>

Thanks a lot for your reply. I looked the Unity3D example and source code of [log\_returns()](https://docs.ray.io/en/latest/_modules/ray/rllib/env/policy_client.html?highlight=multiagent_done_dict) but still confused.  
In the Unity3D example, multiagent\_done\_dict is set as terminateds. But for some customized play, terminateds and truncateds may both need to be feedbacked. I didn’t find any docs to demonstrate how to appropriately define multiagent\_done\_dict.  
Also, I searched multiagent\_done\_dict in the code but didn’t find its application in policy\_server side for training. How are they received and set for reward calculation?  
Could you please demonstrate more about the detail.