Can Ray Serve handle https? [2023]

Ray Libraries (Data, Train, Tune, Serve) Ray Serve
1

Critical for taking app into production:

  • High: It blocks me to complete my task.

There was an old post already for this, has anything changed or anyone know some smart workaround sticking to Ray only and not using a Reverse Proxy?

2

Hi @salman-moh

Yes, Ray Serve by default will start an HTTP proxy on every node (given it has at least one replica or is on head node) API doc: ray.serve.start — Ray 2.7.1

You can read more on how to use it Set Up FastAPI and HTTP — Ray 2.7.1

Also as a side node, if you have a need for sending gRPC requests for slightly better performance over HTTP, there is also a guide on using Serve’s gRPC server Set Up a gRPC Service — Ray 2.7.1

3

I’m talking about HTTPS, not HTTP.

4

Oh sorry, I misread it. The state of HTTPS has not changed since. Feel free to drop a PR for updating that logic. I think you would mostly be changing this file and some API to pass the certs

5

@Gene Can you use TLS configuration for all HTTP comms? TLS Authentication — Ray 2.7.1. This is more for K8s or use if for secure gRPC?

6

Thanks for the doc Jules! I think the doc is about using TLS between Ray clusters. The question is more about how the HTTP proxy works in Ray Serve. In the current state, Ray Serve uses still uses uvicorn server under the hood and requires some code changes to get that TLS setup for HTTPS :slightly_smiling_face:

7

Yes, indeed @gene. Wonder if we should file a issue if there is more demand for it. Surprised that it has not come up.

8

Did some search and found there is an issue related to this already [Serve] Allow HTTPs Options in Ray Serve · Issue #26814 · ray-project/ray · GitHub I think we can prioritized accordingly

1 Like
9

@salman-moh There’s issue filed already, and we as team will triage it to prioritize it. Can we mark this as resolved?