Ray clusters typically keep the head node running, and some system actors (like stats or dashboard actors) may remain alive and can be scheduled on GPU nodes, preventing full GPU scale-down even when min replicas are set to 0. This is a known behavior, especially if system actors are not restricted to CPU-only nodes, causing GPU nodes to stay alive unnecessarily after user jobs finish. Manual intervention (e.g., killing lingering actors) is sometimes needed to release GPU resources completely, as discussed in community threads and code examples (How does Ray actor work?).
To avoid this, ensure system actors are scheduled on CPU nodes or configure the head node to not claim GPU resources. In Kubernetes, set num-gpus: "0" for the head node in your RayCluster spec to prevent the head from reserving GPUs (Troubleshoot multi-node GPU serving on KubeRay). Would you like more detail on how to configure this or how to identify and kill lingering actors?
Sources:
Hint: Mention @Herald in the post for followups.