# Ray Data Performance Issues

**URL:** <https://discuss.ray.io/t/ray-data-performance-issues/4742>\
**Category:** Ray Data\
**Created:** [January 19, 2022, 6:32pm UTC](https://discuss.ray.io/t/ray-data-performance-issues/4742 "2022-01-19T18:32:29Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Preeti\_Joshi](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/preeti_joshi/32/1399_2.png) [@Preeti\_Joshi](https://discuss.ray.io/u/Preeti_Joshi)\
**Post date:** [January 19, 2022, 6:32pm UTC](https://discuss.ray.io/t/ray-data-performance-issues/4742/1 "2022-01-19T18:32:29Z")

</div>

Hi Ray team,

We are trying to

- Load a large dataset (4 Billion plus rows)
- Shard it across a ray cluster
- Perform a task (like executing a query) on each shard
- Get the results from each shard
- Combine the results

To achieve this we have tried the following so far

- Cluster config: 4 nodes (16 cores (CPU) , 64 GB Memory and 1TB of object store)
- Stateful Actors for data load attached to a placement group with a STRICT\_SPREAD strategy - guaranteeing one worker per shard
- Perform execute task (Actor Method) using the same worker
- Perform another ray task to collect the results

Questions:

1. With the following placement\_group settings

```auto
bundles =[{"CPU": 1}, {"CPU": 1}, {"CPU": 1}, {"CPU": 1}]
strategy="STRICT_SPREAD"
num_cpus=4

```

Is this the optimum configuration? How do we utilize all the available cores?  
Currently it is not clear whether the above config is able to achieve that.

1. Will bumping up the CPUs like below in each bundle help?

```auto
bundles =[{"CPU": 2}, {"CPU": 2}, {"CPU": 2}, {"CPU": 2}]
strategy="STRICT_SPREAD"
num_cpus=8

```

1. We observe slowness while collecting the results.

```auto
@remote

def collect_results:
       # code to collect the results from each shard

```

How does ray ensure that the task utilizes all available cores on the given worker?

Please review and provide your feedback on this.

Thanks,  
Preeti

---

<div class="post-metadata">

**Author:** ![Preeti\_Joshi](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/preeti_joshi/32/1399_2.png) [@Preeti\_Joshi](https://discuss.ray.io/u/Preeti_Joshi)\
**Post date:** [January 25, 2022, 12:31pm UTC](https://discuss.ray.io/t/ray-data-performance-issues/4742/2 "2022-01-25T12:31:17Z")

</div>

@Alex , @Clark_Zinzow , @sangcho can you please provide your feedback/pointers on the performance issues mentioned above? Any guidelines on tuning the performance would be very helpful.