To properly release Ray object store memory in your case, ensure that all ObjectRefs (such as those for numpy arrays or big lists) are deleted or go out of scope as soon as you no longer need them. This includes removing references from dictionaries, variables, and any Python data structures. After deleting the ObjectRefs, you can optionally call gc.collect() to force Python garbage collection, but the key is to remove all references so Ray’s distributed reference counting can evict the objects from the object store (Ray memory management, Ray memory debugging guide).
If you are using Ray Serve and passing ObjectRefs between deployments, make sure to dereference (i.e., await or ray.get) and then delete them as soon as possible. Avoid holding onto large objects or ObjectRefs in global variables or long-lived objects. Use the ray memory command to check for lingering references that may be pinning objects in the store. If you still see memory not being released, check for nested references or objects captured inside other objects (Ray memory debugging guide).
Would you like a code example or more detail on using ray memory to debug?
Sources:
- https://github.com/ray-project/ray/issues/20618
- https://discuss.ray.io/t/object-store-spilling-terabytes-of-data/8699
Hint: Mention @RunLLM in the post for followups.