# Very slow run and only nan results when using cluster of 64 cpus
**URL:** https://discuss.ray.io/t/very-slow-run-and-only-nan-results-when-using-cluster-of-64-cpus/9581
**Category:** RLlib
**Created:** [March 1, 2023, 11:03pm UTC](https://discuss.ray.io/t/very-slow-run-and-only-nan-results-when-using-cluster-of-64-cpus/9581 "2023-03-01T23:03:21Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@george\_sk](https://discuss.ray.io/u/george_sk)
#### Post date: [March 1, 2023, 11:03pm UTC](https://discuss.ray.io/t/very-slow-run-and-only-nan-results-when-using-cluster-of-64-cpus/9581/1 "2023-03-01T23:03:21Z")
**How severe does this issue affect your experience of using Ray?**
- High: It blocks me to complete my task.
Hello, I am using the following code that works ok in my laptop with both both linux and windows operating systems. But when I am trying to run it on a cluster of 64 CPUs, it is extremely slow compared to my laptop and it produces only nan as episode reward, while episodes\_this\_iter=0 for every iteration. In my laptop I have been using num\_cpus\_for\_local\_worker=10 and num\_rollout\_workers=2. Also I get the warning:
> The maximum number of pending trials has been automatically set to the number of available cluster CPUs, which is high (140 CPUs/pending trials). If you’re running an experiment with a large number of trials, this could lead to scheduling overhead. In this case, consider setting the `TUNE_MAX_PENDING_TRIALS_PG` environment variable to the desired maximum number of concurrent trials.
while the CPUs I use are 64 (so is ray seeing 140?). I tried setting `TUNE_MAX_PENDING_TRIALS_PG` to 1 and 30, but there was no difference.
I use ray 3.0.0 dev0, because it is the only version that currently supports waterworld.v4.
```auto
from ray import air, tune
from ray.tune.registry import register_env
from ray.rllib.algorithms.ppo import PPOConfig
from ray.rllib.env.wrappers.pettingzoo_env import PettingZooEnv
from pettingzoo.sisl import waterworld_v4
if __name__ == " __main__":
# RDQN - Rainbow DQN
# ADQN - Apex DQN
for i in range(9,11):
def env_creator(args):
return PettingZooEnv(waterworld_v4.env(n_pursuers=5, n_evaders=5))
env = env_creator({})
register_env("waterworld", env_creator)
obs_space = env.observation_space
act_spc = env.action_space
policies = {"shared_1": (None, obs_space, act_spc, {})
# "shared_2": (None, obs_space, act_spc, {})
# "pursuer_5": (None, obs_space, act_spc, {})
}
config = (
PPOConfig()
.environment("waterworld")
.resources(num_gpus=0, num_cpus_for_local_worker=32)
.rollouts(num_rollout_workers=30) # default = 2 (I should try it)
.framework("torch")
.multi_agent(
policies=policies,
policy_mapping_fn=(lambda agent_id, *args, **kwargs: "shared_1"),
)
)
tune.Tuner(
"PPO",
run_config=air.RunConfig(
name="waterworld_v4 n5 shared trial {0}w".format(i),
stop={"training_iteration": 1500},
checkpoint_config=air.CheckpointConfig(
checkpoint_frequency=10,
),
),
param_space=config.to_dict(),
).fit()
```
Please let me know if I should use any other different resources setting, or there are other slutions I could try.
Thanks, George
---
### Author:  [@mannyv](https://discuss.ray.io/u/mannyv)
#### Post date: [March 2, 2023, 12:28pm UTC](https://discuss.ray.io/t/very-slow-run-and-only-nan-results-when-using-cluster-of-64-cpus/9581/2 "2023-03-02T12:28:21Z")
Hi @george_sk,
I think you num\_sgd\_iter st have a lot of policies, a large training batch, and a lot of sgd iterations. You will probably need to tune (not the library) those for your resources.
As a test, how does it change if you set the training argument num\_sgd\_iter to 1? That is probably lower than you will ultimately want but it is good for a sanity test.
---
### Author:  [@george\_sk](https://discuss.ray.io/u/george_sk)
#### Post date: [March 14, 2023, 1:28pm UTC](https://discuss.ray.io/t/very-slow-run-and-only-nan-results-when-using-cluster-of-64-cpus/9581/3 "2023-03-14T13:28:51Z")
Hi @mannyv . When setting sgd\_iterations\_1 it runs faster, but I don’t think faster than my laptop still. I also put num\_rollout\_workers=0 and num\_cpus\_for\_local\_worker=64 but still it doesnt seem much faster. What is the factor that makes it run slowly? Is it the large number of num\_rollout\_workers or something else? I will try ray 2\_3\_0 and gpu as well. But setting num\_sgd\_iter=1 is not ideal since it is a learning parameter that can affect efficiency I suppose. For example the weird thing is that in my laptop I use num\_rollout\_workers=2, num\_cpus\_for\_local\_worker=10 and the preset num\_sgd\_iter (I think it i 30) and this is way faster than using num\_rollout\_workers=2 and num\_cpus\_for\_local\_worker=62 and preset num\_sgd\_iter on the cluster (hpc). Do you have any idea why this is happening?
Thanks.
---
### Author:  [@george\_sk](https://discuss.ray.io/u/george_sk)
#### Post date: [March 14, 2023, 1:52pm UTC](https://discuss.ray.io/t/very-slow-run-and-only-nan-results-when-using-cluster-of-64-cpus/9581/4 "2023-03-14T13:52:33Z")
Also something I forgot. I have read that one solution might be to set TUNE\_MAX\_PENDING\_TRIALS\_PG to 1 or the number of avalable cpus. But even though I set it on the environment I still get the warning:
> The maximum number of pending trials has been automatically set to the number of available cluster CPUs, which is high (140 CPUs/pending trials). If you’re running an experiment with a large number of trials, this could lead to scheduling overhead. In this case, consider setting the `TUNE_MAX_PENDING_TRIALS_PG` environment variable to the desired maximum number of concurrent trials.
So it seems it does not rcognise the value I set. Is there a way to set it internally through an RLlib variable? Because I can’t find something relevant.
---
### Author:  [@mannyv](https://discuss.ray.io/u/mannyv)
#### Post date: [March 14, 2023, 2:28pm UTC](https://discuss.ray.io/t/very-slow-run-and-only-nan-results-when-using-cluster-of-64-cpus/9581/5 "2023-03-14T14:28:12Z")
Hi @george_sk,
One quick note. Changing the num\_cpus\_per\_… Should not have any affect on performance. This is because the num\_cpus are only used for book keeping in ray. That controls how many workers can run simultaneously which increases parallelism during the sampling of new data phase. But within each worker adding more cpus does not affect parallelism in any way.
---
### Author:  [@george\_sk](https://discuss.ray.io/u/george_sk)
#### Post date: [March 14, 2023, 4:15pm UTC](https://discuss.ray.io/t/very-slow-run-and-only-nan-results-when-using-cluster-of-64-cpus/9581/6 "2023-03-14T16:15:37Z")
Hi @mannyv . Sorry if it is naive but I am new to parallelism. Increasing parallelism should improve performance right? And could also increase data samples so it could lead to bottlenecks in some cases I suppose. I understand what you say about num\_cpus\_per since it is on rollout workers that do the sampling. But is it the same for this?
> **num\_cpus\_for\_local\_worker** – Number of CPUs to allocate for the algorithm.
Because I understand from the documentation that it allocates more cpus to the worker running the algorithm, so it should increase performance. I haven’t used number\_cpus\_per… so far.
Generally how could I utilise all 64 CPUs and what RLlib parameters should I use?
Should I use **num\_trainer\_workers**?
Thanks