Multiple node types with Ray/Kubernetes

Ray Clusters Kubernetes
1

We’ve successfully launched Ray Autoscaler on Kubernetes on AWS. However, we’d like to have ray autoscaler support multiple instance types (similar to how the node types work when running the autoscaler natively on AWS). For example, we’ve created node selectors for instance types in k8s via:

# c5.2xlarge
nodeSelector:
  beta.kubernetes.io/instance-type: c5.2xlarge
# p2.xlarge
nodeSelector:
  beta.kubernetes.io/instance-type: p2.xlarge

However, we are unsure how to link those to ray custom resources, such that if I call an actor with resources p2.xlarge=1, ray will trigger EKS to autoscale the pods to fill those demands. Is that supported feature, and how do we enable that in the config?

2

@Dmitri can you help take a look?

3

@Hanlin_Tang
This is indeed a supported feature – we’re working on improving the docs.
Multiple node-type configs work on Kubernetes: just fill in the node_config field for each node_type with the configuration for a K8s pod

.
.
.
available_node_types:
  type_a:
    resources: {x:1, y:2}
    node_config:
      apiVersion: v1
      kind: Pod
      metadata:
        generateName: worker-type-a
.
.
.
4

Bumping this up, docs do not seem to have info for this extremely useful feature.

@Dmitri can you explain where exactly do we put this for KubeRay clusters? Specifically deployed with Helm?