Does memory-aware scheduling use real-time value?

Ray Core
1

When I specify memory request for a task/actor, does scheduler take into account the current free heap memory cluster-wide, or it simply subtracts already submitted requests and makes decision based on the reminder?

If later, is it possible to tell the scheduler to monitor memory in real-time?

Also when specifying memory, does it mean heap-memory or obj-store memory (or combined)? Is it possible to request both (i.e. I know my tasks heap usage and how much it will output to obj store), so scheduler knows what to submit without OOMs?

2

Unfortunately the memory argument for ray.remote is only a logical request, so it has no connection to the real-time memory usage. See here for more info on this feature. We’re actively working on ways to improve real-time heap memory usage, see here for details on preventing out-of-memory errors.