I am wondering how does Ray handle task processes when task is finished. When I look at RAM usage it seems as RAM isn’t released fully after completing task(only around 30%), it looks like Ray reuses processes from previous tasks. Can someone explain me how does this function, or am I doing something wrong. My program runs but I am concerned with RAM usage.
You have to also check SHR usage (you can see from htop). The actual RAM usage is RES - SHR (because Ray uses SHR for object store, and it is not GC’ed from the worker until the object is actually gone for performance reason).
The tasks are scheduled on worker processes that could be reused.