If we start Ray Serve and create a deployment in one script/process (ray.init(address = “ray://MACHINE_IP:10001”)), then it should be possible to list the Serve deployments from another script/process and get hold of the handle, given that we initialise Ray with the same ray address.
But instead we are getting the following error:
ray.serve.exceptions.RayServeException: There is no instance running on this Ray cluster. Please call `serve.start(detached=True) to start one.
I am sure there is something we are missing here, any help would be appreciated.
I agree, this should definitely be working if you are connected to the same cluster. Could you please give us a script that will reproduce the issue?
Figured out the issue. In the code below it works if I run ray.start(detached=True). If I set detached=False it does not work. Behaviour makes sense, but I think it is worth to mention this a bit more explicitly in the documentation.
The code for ray.serve.api states:
“If calling from the driver program, the Serve instance on this Ray cluster must first have been initialized using
import ray.serve as serve
# Take the message to return as an argument to the constructor.
def __init__(self, msg):
self.msg = msg
def __call__(self, request):
def other_method(self, arg):
# Serve will be shut down once the script exits, so keep it alive manually.