Disable route for Serve Backend

Hi everyone,

is it still possible to serve backends without a route with the new deployment api?
The docs state that the default is /{name}

route_prefix ( Optional [ str ] ) – Requests to paths under this HTTP path prefix will be routed to this deployment. Defaults to ‘/{name}’.

Before if you did not provide a route it was not available over HTTP if I got it correct.

So if I do not want the backend to be only available via python code, can I still disable the HTTP route? And if yes, how would I do that?

Best,
Tanja

@TanjaBayer your understanding here is correct, there’s currently no way to prevent the deployment from being available over HTTP. This should be relatively easy to add, let me make an issue and we can prioritize it as it seems generally useful.

For my own understanding, what is the specific reason why you want to disable the deployment from being available over HTTP?

The way we are doing the backend deployment is that we are setting up a graph-like structure where only the starter nodes (backends) should be able to be triggered via HTTP (only they are secured via authentication). If all intermediate nodes are also available via HTTP you need to ensure no one is calling them by accident or when trying to hack your system.

Theoretically someone could also call the python endpoints which are not the starter nodes, but that is much less likely than for the ones which are available via HTTP. Also because you are able to query all available routes via <url>/-/routes. And they are only available form inside the ray cluster, while the HTTP endpoints will be exposed to the world.

The workaround would be to setup the nginx/ingress in that way that only the relevant endpoints can be reached, however that would be only a kind of workaround in my opinion. Not enabling them straight from the beginning would be the nicer way if that does not involve a lot of overhead in other parts of the implementation logic.

3 Likes