Unexpected Subprocess behaviour inside a loop in a ray method

2

Hi Pooja,

for me the following code works nicely:

import subprocess
import ray

ray.init()

@ray.remote
def get_info(val):
  cmd = f"echo {val}"
  out=subprocess.check_output(cmd, shell=True)
  return out

for i in range(0,1000):
  future = get_info.remote(i)
  print(ray.get(future))

Are you using a homogeneous cluster for processing? What kind of machines? Are you sure each machine supports the command you call? If it is a custom script, you should probably use absolute paths to call it.

By the way, the way you write your code it is not parallelized. Is that what you want? If you want parallel execution, you can do something like this:

futures = [get_info.remote(i) for i in range(1000)]
print(ray.get(futures))