# Target\_network\_update\_freq APEX vs DQN

**URL:** <https://discuss.ray.io/t/target-network-update-freq-apex-vs-dqn/6328>\
**Category:** RLlib\
**Created:** [May 30, 2022, 8:39am UTC](https://discuss.ray.io/t/target-network-update-freq-apex-vs-dqn/6328 "2022-05-30T08:39:16Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Nicolas\_Carrara](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/nicolas_carrara/32/2541_2.png) [@Nicolas\_Carrara](https://discuss.ray.io/u/Nicolas_Carrara)\
**Post date:** [May 30, 2022, 8:39am UTC](https://discuss.ray.io/t/target-network-update-freq-apex-vs-dqn/6328/1 "2022-05-30T08:39:16Z")

</div>

**How severe does this issue affect your experience of using Ray?**

- Low: It annoys or frustrates me for a moment.

I want to update my target network every 1000 steps. One episode lasts for 1000 steps. So I expect 1 update per episode.

On DQN it is straightforward, I set `target_network_update_freq=1000` and that’s it.

On APEX-DQN, it depends on a lot of parameters, namely `train_batch_size`, `training_intensity`, `num_workers` and ofc `target_network_update_freq`.

How can I define properly those parameters to get 1 update per episode fed to the learner on APEX? The documentation does not say anything about it.

See the tensorboard metric bellow where I’ve tried different combination of parameters for APEX, vs DQN (in teal)

![apex_vs_dqn](https://us1.discourse-cdn.com/flex020/uploads/ray/original/2X/e/eca71394db819e58c801601d09d7ee28573b79ac.png)

---

<div class="post-metadata">

**Author:** ![Nicolas\_Carrara](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/nicolas_carrara/32/2541_2.png) [@Nicolas\_Carrara](https://discuss.ray.io/u/Nicolas_Carrara)\
**Post date:** [May 30, 2022, 9:16am UTC](https://discuss.ray.io/t/target-network-update-freq-apex-vs-dqn/6328/2 "2022-05-30T09:16:15Z")

</div>

I have upgraded to the last commit on master and ray wheel 3.0 and it works as excepted