ServeHandle with named deployment

How severe does this issue affect your experience of using Ray?

  • Medium: It contributes to significant difficulty to complete my task, but I can work around it.

I’m using deployments for models from a model registry (MLflow). I’m using the example code from the docs. I have multiple models, and my deployments are named.

class MLflowDeployment:
    def __init__(self, model_uri):
        self.model = mlflow.pyfunc.load_model(model_uri=model_uri)

    async def __call__(self, request):
        return self.model.predict(...)


I’d like to use the ServeHandle feature with those deployments.
How can I get the handle with named deployments? I looks like the get_handle method doesn’t have a name parameter. Did I miss something?

handle = MLflowDeployment.get_handle()  # missing name param?

By looking into the get_handle method, I see it uses the internal API. As a workaround, should I call it directly with my name?

get_global_client().get_handle(self._name, missing_ok=True, sync=sync)
get_global_client().get_handle("my_mlflow_model_1", missing_ok=True, sync=sync)

Thank you, Jonathan

EDIT: the same question applies to delete a deployment by name.

Try serve.get_deployment(NAME).get_handle()

Thanks, that’s what I was missing. It works well!