Example docker compose to run RayServe app

Ray Serve
1

1. Severity of the issue: (select one)

Medium: Significantly affects my productivity but can find a workaround.
High: Completely blocks me.

2. Environment:

  • Ray version: 2.48
  • Python version: 3.9
  • OS: osx/linux
  • Cloud/Infrastructure:
  • Other libs/tools (if relevant):

3. What happened vs. what you expected:

  • Expected: docker-compose that brings up ray cluster running ray-serve app.
  • Actual: My try is not using the workers it seems.

@RunLLM Need example docker-compose.yml for setting up head node, 2 workers and a ray-serve node that uses head node as proxy. Is using a ray-serve node that runs the ray-serve app the right approach? Is there a better way?

Thanks!

2

To set up a Ray cluster with Docker Compose (head node, 2 workers, and a Ray Serve app), you should run the head and worker nodes as separate services, then deploy your Ray Serve app connected to the head. The Ray Serve app should connect to the head node using ray.init(address='ray://ray-head:10001') and start Serve with serve.start(detached=True, http_options={"host": "0.0.0.0"}) to expose it for external access. This is a standard and recommended approach for local development and testing with Docker Compose, as shown in this example.

A minimal docker-compose.yml would define services for the head, workers, and app, each with the appropriate ports and network settings. The Ray Serve app should not run its own head node but connect to the cluster’s head. For a full example and best practices, see the detailed setup and scripts in this discussion. Would you like a step-by-step breakdown or a sample docker-compose.yml?

Sources:

Hint: Mention @RunLLM in the post for followups.