Public IP was not assigned when utilizing own subnet on Public Cloud

Ray Clusters
1

Public IP was not assigned to EC2 instance (Header node) when I tried to utilize own subnets on AWS . It was continously showing “Not yet available, retrying in 5 seconds”. Once I attached the public IP to the instance, the “ray up” process move forwarded.

Should I mention the public IP assignment in “Ray.yaml”?

  • Result of “ray up” command
AWS config
  IAM Profile: ray-head-v1-instanceprofile
  EC2 Key pair (all available node types): ray-autoscaler_7_us-west-2 [default]
  VPC Subnets (ray.head.default): subnet-08a1722099084c81e
  VPC Subnets (ray.worker.default): subnet-0f4622b4f638bbafb, subnet-07246a792842fab50
  EC2 Security groups (all available node types): sg-0347344f04adba7df [default]
  EC2 AMI (all available node types): ami-0a2363a9cff180a64

Updating cluster configuration and running full setup.
Cluster Ray runtime will be restarted. Confirm [y/N]: y

<1/1> Setting up head node
  Prepared bootstrap config
  New status: waiting-for-ssh
  [1/7] Waiting for SSH to become available
    Running `uptime` as a test.
    Waiting for IP
      Not yet available, retrying in 5 seconds
      Not yet available, retrying in 5 seconds
      Not yet available, retrying in 5 seconds
      Not yet available, retrying in 5 seconds
      Not yet available, retrying in 5 seconds
      Not yet available, retrying in 5 seconds
      Not yet available, retrying in 5 seconds
      Not yet available, retrying in 5 seconds
      Not yet available, retrying in 5 seconds
      Not yet available, retrying in 5 seconds

  • Ray.yaml
available_node_types:
    ray.head.default:
        resources: {}
        node_config:
            InstanceType: m5.large
            ImageId: ami-0a2363a9cff180a64 
            BlockDeviceMappings:
                - DeviceName: /dev/sda1
                  Ebs:
                      VolumeSize: 100
            IamInstanceProfile:
                Arn: arn:aws:iam::787731253944:instance-profile/ray-head-v1-instanceprofile
            # The forllowing subnets are Public subnets.
            SubnetIds: [subnet-08a1722099084c81e, subnet-0b6d69e5b3de492b9]
            
    ray.worker.default:
        min_workers: 6
        max_workers: 10
        resources: {}
        node_config:
            InstanceType: m5.large
            ImageId: ami-0a2363a9cff180a64 
            InstanceMarketOptions:
                MarketType: spot
            IamInstanceProfile:
                Arn: arn:aws:iam::787731253944:instance-profile/ray-worker-v1-instanceprofile
            # The forllowing subnets are Private subnets.
            SubnetIds: [subnet-0f4622b4f638bbafb, subnet-07246a792842fab50]
2

I think @Dmitri / @Alex might be able to help with this one.

3

@AtsushiSugai yeah, ray cluster launcher needs a public ip address to setup the head node

4

Another option (required in some contexts) would be to run ray up from a bastion host: