Worker nodes not available with manual configuration

Ray Core
1

Following these instructions: Launching Cloud Clusters — Ray v2.0.0.dev0

I have a coordinator server running, and also have the IPs of worker machines registered.

However, those workers are not being used for running workloads. The coordinator server prints out ‘terminated’ as their state during startup of the coordinator server.

Is there something special i maybe need to do on the workers so they show up in the cluster and can handle work?

2

i also tried out the ‘manually managed’ option of entering the IPs for the head and worker nodes into the cluster yaml, which also doesn’t seem to work. Am I missing an important step?

I’m getting this as output when deploying the cluster yaml (which looks good, it includes the IPs of worker nodes). However, when running tasks and printing out on which host they get executed, it’s always only the head node that gets printed out>

ClusterState: Loaded cluster state: [‘10.242.132.5’, ‘10.242.132.6’, ‘10.242.132.7’, ‘10.242.132.8’, ‘158.176.183.190’]

3

cc @Ameer_Haj_Ali Can you answer his question?

4

CC @Dmitri , can you please help?

5

I get the sense the functionality in question hasn’t received much attention in a while.

I’m a bit caught up with other things at the moment – in the meantime, I’ve started an issue to track progress on this:

1 Like
6

@Dmitri has there been any update on this?