Ray cluster worker port

Ray Clusters
1

How severe does this issue affect your experience of using Ray?

  • Medium: It contributes to significant difficulty to complete my task, but I can work around it.

I want to know which port does worker use to bind with master node. Can we give any static port here like raylet process start on 6379 in master.?

I need this to monitor worker node availability from our external agent. I formed on-premise ray cluster using cli.

2

Hi @Jules_Damji @Chen_Shen , Could anyone assist me in this

3

Hi @Jules_Damji ., Can you help me in this?. Random ports getting bind with master gcs process port 6379.

4

@BalajiSelvaraj10 The default port is 6379 that worker nodes bind to head node. But you can specify other non-default port with ray start --address=<head-node-address:port>

What precisely are you trying to do?

Does these docs help? Launching an On-Premise Cluster — Ray 2.8.0

5

Hi @Jules_Damji ,

Default port is 6379 for head node raylet process which is /home/xxx/site-packages/ray/core/src/ray/raylet/raylet

When connecting worker node with head node using “ray start --address=head-node-address:port”, some random port running the same process /home/xxx/site-packages/ray/core/src/ray/raylet/raylet in worker node. The port on which the raylet process running in worker node is connected with head node’s 6379 using tcp.

I want to know, could we use static port in worker to bind with head node’s 6379.?

6

@Jules_Damji could u help me in this., This stopped ray provisioning at my end.

7

Hi @Jules_Damji ., Could you assist me in this or can you loop in someone to check this :slightly_smiling_face:

8

Hi @RayAdmin ., Can you check my request and help me out

9

Hi @Chen_Shen ., Can you look into this

10

https://docs.ray.io/en/latest/ray-core/configure.html#all-nodes

The following options specify the range of ports used by worker processes across machines. All ports in the range should be open.

  • --min-worker-port: Minimum port number worker can be bound to. Default: 10002.
  • --max-worker-port: Maximum port number worker can be bound to. Default: 19999.

Port numbers are how Ray disambiguates input and output to and from multiple workers on a single node. Each worker will take input and give output on a single port number. Thus, for example, by default, there is a maximum of 10,000 workers on each node, irrespective of number of CPUs.

In general, it is recommended to give Ray a wide range of possible worker ports, in case any of those ports happen to be in use by some other program on your machine. However, when debugging it is useful to explicitly specify a short list of worker ports such as --worker-port-list=10000,10001,10002,10003,10004 (note that this will limit the number of workers, just like specifying a narrow range).

I think this is the only way to do it.

1 Like