Running Multiple Ray Heads on Same Node - Safety & Best Practices?

Ray Serve
1

Hello,

I’m working on deploying multiple Ray Serve applications with head nodes residing on the same physical node on a system where I don’t have access to Docker. In the ideal scenario, I would be able to do this without containerization, however I can make use of Apptainer containers (formerly Singularity, commonly used in HPC environments) to virtualize the filesystem and potentially the network.

In my setup, ray workers all are deployed on separate nodes, so only overlap is on the head. I also don’t use the ray object store, so /dev/shm shouldn’t be touched at all by Ray. I’ve seen the warnings about not running multiple Ray heads on the same machine, but I’m trying to understand the specific conflict points to ensure my setup is safe.

Question: Besides /tmp and ports, what other resources do multiple Ray heads conflict on? The docs warn against this but don’t detail the specific conflict points.
Specifically wondering about:

  • Does Ray use shared memory, file locks, or IPC beyond /tmp/ray (ignoring ray object store)? Would setting different —temp-dir values suffice for filesystem isolation?
  • Is network isolation actually necessary if ports can be configured to be disjoint?
  • Any Ray Serve-specific shared resources?

Thanks!