How severe does this issue affect your experience of using Ray?
- None: Just asking a question out of curiosity
- Low: It annoys or frustrates me for a moment.
Hello,
I have long running process, where lots of calculations happen. I am actually reading csv file with around 25K rows of data and processing it. I have similar runs for different files. Each run is sent to ray.remote and the calculation happens. So, the bigger the file, the more time it takes. It takes 3-4 hrs to finish. The other process finish faster and return back the data. So the other cpus are idle. Only one process, processing this huge data is running. Can ray share its work to other cpus??
My entire cluster is sitting idle as all processes have finished the tasks assigned except this long running one.
please help me.
Thanks
Amitha