ServeHandle for deployment created with a function with query_params?

Ray Libraries (Data, Train, Tune, Serve) Ray Serve
1

How severe does this issue affect your experience of using Ray?

  • Low: It annoys or frustrates me for a moment.

When I tried combining what’s written in the documentation, I encountered into an issue related to ServeHandle for a deployment created with a function.

I created the following Serve Deployment, which is simplified one of the router example in the doc

import ray
from ray import serve

ray.init(address="auto", namespace="serve")
serve.start(detached=True)


@serve.deployment
def router(request):
    txt = request.query_params["txt"]
    return txt


router.deploy()

I could confirm with HTTP request with txt as a parameter.

And I also tried to access to this deployment via Python, but I couldn’t find a way to do that. I read a best practice to separate a method to call using ServeHandle like:

@serve.deployment(route_prefix="/api")
class Deployment:
    def say_hello(self, name: str):
        return f"Hello {name}!"

    def __call__(self, request):
        return self.say_hello(request.query_params["name"])

It worked perfectly, but I’m wondering if there’s a way to send a request via ServeHandle for deployment created with a function.

Thank you in advance!

2

@nakamasato I would recommend that you use a separate method in a class for this use case (as you figured out!).

1 Like
3

@eoakes Thank you for your super quick reply! I’ll use the best practice!
It seems that supporting a function with the @serve.deployment annotation is for a simple use case, and if there’s more requirements like to support multiple access ways with parameters, it’s recommended to use a class. Is my understanding correct?

4

Yep, that’s exactly right. The function use case is mostly there for simple examples :slight_smile:

1 Like
5

Thank you for your confirmation!