# Struggling tuning Soft Actor Critic

**URL:** <https://discuss.ray.io/t/struggling-tuning-soft-actor-critic/12505>\
**Category:** RLlib\
**Created:** [October 18, 2023, 4:00pm UTC](https://discuss.ray.io/t/struggling-tuning-soft-actor-critic/12505 "2023-10-18T16:00:37Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![sergioval](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/sergioval/32/5133_2.png) [@sergioval](https://discuss.ray.io/u/sergioval)\
**Post date:** [October 18, 2023, 4:00pm UTC](https://discuss.ray.io/t/struggling-tuning-soft-actor-critic/12505/1 "2023-10-18T16:00:37Z")

</div>

**How severe does this issue affect your experience of using Ray?**

- Low: It annoys or frustrates me for a moment.

Hello, I am trying to use SAC with a custom toy environment. PPO solves it very efficiently and robustly, achieving optimal performance right away. However, I am struggling tuning SAC.

SAC achieves optimal episode\_reward\_max right away; but the episode\_reward\_min is low, about 30% of the optimal, and it never improves. The mean is 80% of the optimal, meaning that not too many episodes are contributing to the low min reward. This happens for both training and evaluation.

This doesn’t happen for PPO, which obtains optimal values for episode\_reward\_min and episode\_reward\_max efficiently.

I have tried tuning SAC, but I am not able to get rid of the rogue episodes and overcome the low min reward. The best performance is obtained for a low initial alpha of 0.1.

Is this expected behaviour? Could it be due to some stochasticity during evaluation that I should make deterministic?

Thanks!