No space left on device

[Gaddiel_Ouaknin]

How severe does this issue affect your experience of using Ray?

• Medium: It contributes to significant difficulty to complete my task, but I can work around it.

Is there a way to run ray tune with minimal memory requirements and or another way to avoid the problem of running out of memory when pickling?

[arturn]

I’ve moved this to Air. @Yard1

[Yard1]

You should initialize any large object (eg. model) inside the training function itself.

# don't do this

model = LargeModel()

def training_fn():
    model.forward()

# do this

def training_fn():
    model = LargeModel()
    model.forward()