Hi, say I want to dispatch tasks to multiple slurm nodes. Is there any difference between defining the atomic task and defining grouped atomic tasks regarding the communication cost of job scheduling?
1. define the atomic task
@ray.remote(num_cpus=1) def atomic_task(args): some_heavy_work(args)
2. define grouped atomic tasks
@remote(num_cpus=1) def grouped_atomic_tasks(list_of_args): for args in list_of_args: some_heavy_work(args)
list_of_args is a chunk of args manually grouped by the user for each worker.