1. Severity of the issue: (select one)
None: I’m just curious or want clarification.
Low: Annoying but doesn’t hinder my work.
Medium: Significantly affects my productivity but can find a workaround.
High: Completely blocks me.
2. Environment:
Ray version:
Python version:
OS:
Cloud/Infrastructure:
Other libs/tools (if relevant):
3. What happened vs. what you expected:
Actual:
Hi all!
Trying to understand the impetus, reasoning, design of the Replica ranking system. Is there a design doc/Jira to help me understand? What problem or feature is it design to address?
The Ray Serve replica ranking system assigns each replica a unique, contiguous integer rank (from 0 to N-1) to support distributed workloads that require deterministic, rank-based behavior (e.g., sharded models, collective communication, or distributed inference). The system ensures that each running replica has a unique rank, and ranks are reassigned as needed to maintain contiguity during scaling events. This design enables use cases where replicas need to know their position in a group, such as for distributed training or partitioned inference workloads. For more details, see the Ray Serve advanced guide on replica ranks.
There is no explicit design doc or Jira linked in the available sources, but the advanced guide explains the motivation and lifecycle of ranks, including how they are assigned, maintained, and reassigned during scaling or failures. If you need further details or have a specific use case, the Ray team encourages opening a GitHub issue to discuss requirements with the Serve team.
Would you like a step-by-step breakdown of the design and its use cases?