# How to use a custom critic and default actor in PPO?

**URL:** <https://discuss.ray.io/t/how-to-use-a-custom-critic-and-default-actor-in-ppo/20608>\
**Category:** RLlib\
**Created:** [November 14, 2024, 8:55pm UTC](https://discuss.ray.io/t/how-to-use-a-custom-critic-and-default-actor-in-ppo/20608 "2024-11-14T20:55:46Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Matthew\_Giacovelli](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/matthew_giacovelli/32/6893_2.png) [@Matthew\_Giacovelli](https://discuss.ray.io/u/Matthew_Giacovelli)\
**Post date:** [November 14, 2024, 8:55pm UTC](https://discuss.ray.io/t/how-to-use-a-custom-critic-and-default-actor-in-ppo/20608/1 "2024-11-14T20:55:46Z")

</div>

- High: It blocks me to complete my task.

---

<div class="post-metadata">

**Author:** ![Lars\_Simon\_Zehnder](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/lars_simon_zehnder/32/1185_2.png) [@Lars\_Simon\_Zehnder](https://discuss.ray.io/u/Lars_Simon_Zehnder)\
**Post date:** [November 18, 2024, 4:13pm UTC](https://discuss.ray.io/t/how-to-use-a-custom-critic-and-default-actor-in-ppo/20608/2 "2024-11-18T16:13:56Z")

</div>

@Matthew_Giacovelli thanks for posting and welcome to the forum.

For a custom value network you will need to write your own `RLModule`. See [here](https://github.com/ray-project/ray/blob/master/rllib/examples/rl_modules/classes/tiny_atari_cnn_rlm.py) for an example how this is done for Atari.

You can then build this `RLModule` via the `PPOConfig` by defining an `RLModuleSpec` similar to the same example [here](https://github.com/ray-project/ray/blob/master/rllib/examples/rl_modules/custom_cnn_rl_module.py).