ImpliticitFunc Too Large

Ray AIR (Data, Train, Tune, Serve) Ray Tune
1

How severe does this issue affect your experience of using Ray?

  • High: It blocks me to complete my task.

I am running my code on HPC. I have 72 cpu and A100 GPU but I am getting this error:
The actor ImplicitFunc is very large (95 MiB). Check that its definition is not implicitly
capturing a large array or other object in scope. Tip: use ray.put() to put large objects
in the Ray object store.
I really want to believe that I am making a mistake since 95 MiB is funny number for such a computing resource. I already tried ray put and ray get solutions but doesnt work.I put my resnet18 model and called get in the training function. Is there any way to increase this number ?

2

@ElifCerenGok You might share with us some code snippet where you putting the code in the object store, and also how the model is created. Feel that it might be referencing another object that is too large.

cc: @matthewdeng

3

Hi @ElifCerenGok,

Could you share what your script looks like? Are you perhaps capturing any other global state (e.g. the dataloader) in your trainable?