# Ray distributed memory parallelism
**URL:** https://discuss.ray.io/t/ray-distributed-memory-parallelism/10993
**Category:** Ray Core
**Created:** [June 10, 2023, 3:54am UTC](https://discuss.ray.io/t/ray-distributed-memory-parallelism/10993 "2023-06-10T03:54:00Z")
**Posts on this page:** 4
**Page:** 1
### Author:  [@wrios](https://discuss.ray.io/u/wrios)
#### Post date: [June 10, 2023, 3:54am UTC](https://discuss.ray.io/t/ray-distributed-memory-parallelism/10993/1 "2023-06-10T03:54:00Z")
Hi,
When it comes to creating a ray cluster, is there a way to limit the number of workers per node? My remote function generates large amounts of data and I want to avoid having many threads on the same node.
---
### Author:  [@rickyyx](https://discuss.ray.io/u/rickyyx)
#### Post date: [June 15, 2023, 5:15pm UTC](https://discuss.ray.io/t/ray-distributed-memory-parallelism/10993/2 "2023-06-15T17:15:00Z")
Hi @wrios - great question!
In general, each ray node will be started by `ray start` or `ray.init()`, and you could pass “num\_cpus” to specify the logical number of CPUs that this node has. This will be equivalent to the maximal number of workers processes that Ray starts. (Do note that for each ray worker process, there might be multiple threads being used)
And, how are you creating the cluster? I could probbaly provide more specific help if I know that.
---
### Author:  [@wrios](https://discuss.ray.io/u/wrios)
#### Post date: [June 15, 2023, 5:42pm UTC](https://discuss.ray.io/t/ray-distributed-memory-parallelism/10993/3 "2023-06-15T17:42:04Z")
Thank you for your interest in helping. To set up the cluster I am using a similar script as depicted in this [example](https://github.com/NERSC/slurm-ray-cluster/blob/master/submit-ray-cluster.sbatch). Because a remote function can generate large amounts of data before it returns something, as occur in wave simulation where the history might need to be saved, it can fill up the node memory very easily if I have several ray workers in the same node. I was just wondering if there is a way to enforce ray workers to be distributed across nodes so as to avoid this to happen.
---
### Author:  [@Justin\_Coffi](https://discuss.ray.io/u/Justin_Coffi)
#### Post date: [October 20, 2023, 3:53pm UTC](https://discuss.ray.io/t/ray-distributed-memory-parallelism/10993/4 "2023-10-20T15:53:11Z")
I’m just an amateur Ray user. But have you considered a centrally available location for worker spillover and/or consider enabling dask on ray? This way if the worker fills up, the information is dumped to disk and can be picked up again by the same worker (or another worker if the node or worker fails).