# Rllib runs UNBELIEVABLY slow on windows, even on a basic cartpole environment

**URL:** <https://discuss.ray.io/t/rllib-runs-unbelievably-slow-on-windows-even-on-a-basic-cartpole-environment/4027>\
**Category:** RLlib\
**Created:** [November 3, 2021, 1:45pm UTC](https://discuss.ray.io/t/rllib-runs-unbelievably-slow-on-windows-even-on-a-basic-cartpole-environment/4027 "2021-11-03T13:45:15Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![henry\_lei](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/henry_lei/32/1613_2.png) [@henry\_lei](https://discuss.ray.io/u/henry_lei)\
**Post date:** [November 3, 2021, 1:45pm UTC](https://discuss.ray.io/t/rllib-runs-unbelievably-slow-on-windows-even-on-a-basic-cartpole-environment/4027/1 "2021-11-03T13:45:15Z")

</div>

I’m trying to train an agent on a custom environment, but training is so slow even with the default cartpole example, with or without a gpu.

My set up is:  
Windows 10  
Ray 1.3.0  
Tensorflow 2.4  
python 3.7.10

when I run:  
from ray import tune  
from ray.rllib.agents.ppo import PPOTrainer

myconfig = {“env”: “CartPole-v0”}  
myconfig[“num\_workers”] = 12  
myconfig[“num\_gpus”] = 0  
myconfig[“log\_level”] = “WARN”  
#myconfig[“framework”] = “torch” #also tried with tensorflow 1,2, eager  
tune.run(PPOTrainer, config=myconfig)

timers:  
learn\_throughput: 580.592  
learn\_time\_ms: 8267.423  
sample\_throughput: 773.311  
sample\_time\_ms: 6207.078  
update\_time\_ms: 6.689

and the time per iteration is about 15 seconds. I’m mostly concerned about the learn\_time\_ms since, as I understand, that is the sgd update step while the sample\_time\_ms is for trajectory collection (it seems a little slow too?).

---

<div class="post-metadata">

**Author:** ![mickelliu](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/mickelliu/32/1347_2.png) [@mickelliu](https://discuss.ray.io/u/mickelliu)\
**Post date:** [November 16, 2021, 1:53pm UTC](https://discuss.ray.io/t/rllib-runs-unbelievably-slow-on-windows-even-on-a-basic-cartpole-environment/4027/2 "2021-11-16T13:53:26Z")

</div>

did you change your train\_batch\_size and sgd\_minibatch\_size?

---

<div class="post-metadata">

**Author:** ![Lars\_Simon\_Zehnder](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/lars_simon_zehnder/32/1185_2.png) [@Lars\_Simon\_Zehnder](https://discuss.ray.io/u/Lars_Simon_Zehnder)\
**Post date:** [November 17, 2021, 12:59pm UTC](https://discuss.ray.io/t/rllib-runs-unbelievably-slow-on-windows-even-on-a-basic-cartpole-environment/4027/3 "2021-11-17T12:59:42Z")

</div>

@henry_lei ,

`train_batch_size` and `sgd_minibatch_size` would also be my first guess. A second is your number of workers: the number of workers his quite high and that might cause some overload.