Remote function too large - function size error

Ray Clusters
1

Hi,
I’m new to ray and struggle with the function size restriction.
“ValueError: The remote function main.predict is too large (97 MiB > FUNCTION_SIZE_ERROR_THRESHOLD=95 MiB).”
I saw that other people had the same problem and tried to apply the suggested solutions. However, unfortunately I did not manage.
This is a reduced version of my code:

# Start Ray cluster
ray.init(num_cpus=num_cpus, ignore_reinit_error=True)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             

# val is an array representing an image                                   
ref = ray.put(val) 
@ray.remote                                                                                                     
def predict(ref):
pred = classification_model(ray.get(ref)).softmax(0)   # resnet50 model
class_id = preds.argmax().item()

result_refs = [ ]
result_refs.append(predict.remote(ref))
results = ray.get(result_refs)

My code is supposed to parallelize a pipeline for hundreds of images in the end and this is the core of it.
Maybe someone knows a solution?
Cheers!

2

@valentina What’s classification_model()? Does it create the instance of the mode each time predict() is called?

You might want to put the model into the object store, and send a reference to each task to do the prediction. Notice how in this tutorial we put images and model in the object store.

1 Like
4

Thank you very much! I was mainly concentrating on the data so I did not think about the model being the problem, works like a charm now.

5

Excellent @valentina.