Terraform scripts for Ray deployment to AWS EKS

sunita_nair30 · October 24, 2023, 11:06am

How severe does this issue affect your experience of using Ray?

Medium: It contributes to significant difficulty to complete my task, but I can work around it.

Hi Team,

I plan to deploy ray to our EKS environment. I would like to know if there are any Terraform scripts that I can use to deploy ray through CI/CD.

Thank you,
Sunita

JasonHan · November 8, 2023, 4:56pm

AWS had posted a blog on this, and there is an example in there.

github.com

aws-ia/terraform-aws-eks-blueprints/blob/v4.10.0/examples/ai-ml/ray/ray-clusters/example-cluster.yaml

# For most use-cases, it makes sense to schedule one Ray pod per Kubernetes node.
# This is a configuration for a RayCluster with 1 Ray head pod and 1 Ray worker pod.
# Each pod requests 54 Gi memory and 14 CPU.
# Each pod can be scheduled on a virtual machine with roughly 64 Gi memory and 16 CPU.
# (AWS: m5.4xlarge, GCP: e2-standard-16, Azure: Standard_D5_v2)
# Optimal resource allocation will depend on your Kubernetes infrastructure and might
# require some experimentation.
apiVersion: ray.io/v1alpha1
kind: RayCluster
metadata:
  labels:
    controller-tools.k8s.io: "1.0"
    # An unique identifier for the head node and workers of this cluster.
  name: raycluster-autoscaler
  namespace: ${namespace}
spec:
  rayVersion: '1.13.0'
  ######################headGroupSpecs#################################
  # head group template and specs, (perhaps 'group' is not needed in the name)
  headGroupSpec:

This file has been truncated. show original

Not sure why the script was deleted in the later versions, but it can give some clues on how to come up with your own script.

sunita_nair30 · November 15, 2023, 7:21pm

Thank you so much Jason.

wesleyh · April 3, 2024, 7:03am

Not sure why the script was deleted in the later versions, but it can give some clues on how to come up with your own script.

Script moved into a different repo: data-on-eks/ai-ml/ray/terraform at main · awslabs/data-on-eks · GitHub

Topic		Replies	Views
Deploying to Ray Cluster on EKS Kubernetes	0	83	May 2, 2024
Ray Serve Pods Scheduling Failing Ray Serve	3	50	July 26, 2024
Ray cluster question	4	510	January 26, 2021
Kubernetes YAML files for manual deployment Ray Clusters	0	348	August 10, 2021
Resource utilization for RayServe in Kubernetes (AKS) Kubernetes	4	495	June 24, 2022

Terraform scripts for Ray deployment to AWS EKS

Related Topics