Extra processes created with Tensorflow

When a tensorflow operation is called from inside a remote function or any of it callees, processes are created on all cores importing even before the remote function starts executing. Is there a way to do the imports only whenever a remote process is called?

Attaching the code and screenshots of the output
TF Version: 1.14
Ray Version: 1.2.0

def f():
    print("Printing at " + str(time.time()))

if __name__ == "__main__":
    next_idx = 1
    current_objs = [f.remote()]
    while len(current_objs):
        done, current_objs = ray.wait(current_objs)
        if next_idx < 100:
            next_idx += 1

Hey sohamparikh94,

it might be possible to import TensorFlow inside the task like so:

def f():
  import tensorflow as tf

this should make sure tensorflow specific loading is constrained to that task.


Thanks for the response @pcmoritz. In my use case, TensorFlow is imported at the beginning in a separate class which is called from my function and it contains multiple methods that use TensorFlow. So I am not sure if importing it in every method is the best way