Error when trying to get handle to Ray Serve deployment

Ray Libraries (Data, Train, Tune, Serve) Ray Serve
1

If we start Ray Serve and create a deployment in one script/process (ray.init(address = “ray://MACHINE_IP:10001”)), then it should be possible to list the Serve deployments from another script/process and get hold of the handle, given that we initialise Ray with the same ray address.

But instead we are getting the following error:
ray.serve.exceptions.RayServeException: There is no instance running on this Ray cluster. Please call `serve.start(detached=True) to start one.

I am sure there is something we are missing here, any help would be appreciated.

2

I agree, this should definitely be working if you are connected to the same cluster. Could you please give us a script that will reproduce the issue?

3

Figured out the issue. In the code below it works if I run ray.start(detached=True). If I set detached=False it does not work. Behaviour makes sense, but I think it is worth to mention this a bit more explicitly in the documentation.

The code for ray.serve.api states:
“If calling from the driver program, the Serve instance on this Ray cluster must first have been initialized using serve.start(detached=True).”

import time
import ray
import ray.serve as serve

ray.init(address="ray://RAY-HEAD:10001", namespace="hello")
serve.start(detached=True)

@serve.deployment
class MyFirstDeployment:
  # Take the message to return as an argument to the constructor.
  def __init__(self, msg):
      self.msg = msg

  def __call__(self, request):
      return self.msg

  def other_method(self, arg):
      return self.msg

MyFirstDeployment.deploy("Hello world!")

 # Serve will be shut down once the script exits, so keep it alive manually.
while True:
    time.sleep(5)
    print('Alive')