Total concurrent schedule tasks is unexpected

1

I use a simple code such as:
dataset.map_group(my_func, num_cpus=1)

tasks are running in a machine with 15cpu, expected to concurrently run 15 tasks at a time, but i found it’s only 5 nearly tasks running at a time. Is there something else i missing to config?

20241016-170201
20241016-1702013390×1708 291 KB

2

more metrics about the task:

20241016-170206
20241016-1702063286×978 184 KB

3

20241016-170210
20241016-1702103240×1056 215 KB

4

20241016-170214
20241016-1702143286×666 75 KB

5

what is your task resource allocation/configuration looking like?

6

I only set num_cpus=1 for dataset.map_group of ray.data, default value for others

7

What’s the ray and python version? Can you share a script for us to reproduce?