# The hyperparameters for SAC to solve “CartPole-v0”

**URL:** <https://discuss.ray.io/t/the-hyperparameters-for-sac-to-solve-cartpole-v0/4864>\
**Category:** RLlib\
**Created:** [February 1, 2022, 3:00am UTC](https://discuss.ray.io/t/the-hyperparameters-for-sac-to-solve-cartpole-v0/4864 "2022-02-01T03:00:06Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![yuwenche](https://avatars.discourse-cdn.com/v4/letter/y/ecd19e/32.png) [@yuwenche](https://discuss.ray.io/u/yuwenche)\
**Post date:** [February 1, 2022, 3:00am UTC](https://discuss.ray.io/t/the-hyperparameters-for-sac-to-solve-cartpole-v0/4864/1 "2022-02-01T03:00:06Z")

</div>

I follow the following link to set up the hyperparameters for SAC to solve “CartPole-v0”, which is a very easy task. However, the mean rewards always remain about 10. Do I miss something?

> <https://github.com/ray-project/ray/blob/master/rllib/tuned_examples/sac/cartpole-sac.yaml>

the code:

import ray  
#import ray.rllib.agents.ppo as ppo  
import ray.rllib.agents.sac as sac

ray.init()  
config = sac.DEFAULT\_CONFIG.copy()  
config[“num\_gpus”] = 0  
#config[“framework”] = “torch”#  
config[“framework”] = “tf”  
config[“no\_done\_at\_end”] = False  
config[“gamma”] = 0.95  
config[“target\_network\_update\_freq”] = 32  
config[“tau”] = 1.0  
config[“train\_batch\_size”] = 32  
config[“optimization”][“actor\_learning\_rate”] = 0.005  
config[“optimization”][“critic\_learning\_rate”] = 0.005  
config[“optimization”][“entropy\_learning\_rate”] = 0.0001

#trainer = sac.SACTrainer(config=config, env=“MountainCar-v0”)  
trainer = sac.SACTrainer(config=config, env=“CartPole-v0”)

for i in range(5000):  
result = trainer.train()  
if i % 10 == 0:  
#checkpoint = trainer.save()  
print("i: “, i,” reward: ",result[‘episode\_reward\_mean’])

---

<div class="post-metadata">

**Author:** ![gjoliver](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/gjoliver/32/1490_2.png) [@gjoliver](https://discuss.ray.io/u/gjoliver)\
**Post date:** [February 7, 2022, 1:12am UTC](https://discuss.ray.io/t/the-hyperparameters-for-sac-to-solve-cartpole-v0/4864/2 "2022-02-07T01:12:04Z")

</div>

this should work. I just tested it myself while answering another thread.  
actually you can quickly verify this with:

```auto
rllib train -f rllib/tuned_examples/sac/cartpole-sac.yaml 

```

---

<div class="post-metadata">

**Author:** ![sven1977](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/sven1977/32/53_2.png) [@sven1977](https://discuss.ray.io/u/sven1977)\
**Post date:** [February 9, 2022, 10:37am UTC](https://discuss.ray.io/t/the-hyperparameters-for-sac-to-solve-cartpole-v0/4864/3 "2022-02-09T10:37:06Z")

</div>

Hey @yuwenche , I’m with @gjoliver here, this particular test is part of our CI-testing suite, so it’s guaranteed to work (it’s tested on every single PR and we won’t merge stuff into master if these don’t pass).  
Could you send us a small, self-sufficient repro script that reproduces your issue with not learning?  
Thanks

---

<div class="post-metadata">

**Author:** ![yuwenche](https://avatars.discourse-cdn.com/v4/letter/y/ecd19e/32.png) [@yuwenche](https://discuss.ray.io/u/yuwenche)\
**Post date:** [February 10, 2022, 4:13am UTC](https://discuss.ray.io/t/the-hyperparameters-for-sac-to-solve-cartpole-v0/4864/4 "2022-02-10T04:13:08Z")

</div>

Thanks for your replies.  
I have responded in another thread as the following link. Actually, I used two different user names - one with my github, the other for new signup. I am sorry for inconvenience.

[https://discuss.ray.io/t/the-hyperparameters-for-sac-to-solve-cartpole-v0/4909](https://discuss.ray.io/t/the-hyperparameters-for-sac-to-solve-cartpole-v0/4909)