Ray Serve + Ray Client: AttributeError: 'ClientObjectRef' object has no attribute '_on_completed'

Ray Libraries (Data, Train, Tune, Serve) Ray Serve
1

Having some issues getting Ray Serve working with the Ray client. I have Ray head & serve running on my machine. I create an endpoint and I’m able to successfully hit the endpoint via HTTP request and using the server handle.

However, if I’m using the ray client, I get the following error when running get_handle:

Traceback (most recent call last):
  File "./phoenix/benchmarks/request_test.py", line 16, in <module>
    handle = serve.get_handle('remote')
  File "/usr/local/lib/python3.7/site-packages/ray/serve/api.py", line 982, in get_handle
    endpoint_name, missing_ok=missing_ok, sync=sync, _internal=True)
  File "/usr/local/lib/python3.7/site-packages/ray/serve/api.py", line 88, in check
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/ray/serve/api.py", line 605, in get_handle
    router = self._get_proxied_router(sync=sync, endpoint=endpoint_name)
  File "/usr/local/lib/python3.7/site-packages/ray/serve/api.py", line 165, in _get_proxied_router
    endpoint,
  File "/usr/local/lib/python3.7/site-packages/ray/serve/api.py", line 103, in __init__
    self.router = Router(controller_handle, endpoint_tag, self._async_loop)
  File "/usr/local/lib/python3.7/site-packages/ray/serve/router.py", line 219, in __init__
    call_in_event_loop=self._loop,
  File "/usr/local/lib/python3.7/site-packages/ray/serve/long_poll.py", line 62, in __init__
    self._reset()
  File "/usr/local/lib/python3.7/site-packages/ray/serve/long_poll.py", line 72, in _reset
    self._poll_next()
  File "/usr/local/lib/python3.7/site-packages/ray/serve/long_poll.py", line 87, in _poll_next
    self._current_ref._on_completed(
AttributeError: 'ClientObjectRef' object has no attribute '_on_completed'

To recreate, add your endpoint:

import ray
from ray import serve

class SomeClass:

    def __init__(self):
        self.message = 'it works!'

    def __call__(self, request):
        return self.message

ray.util.connect('ip:10001')
serve.connect()

serve.create_backend("remote_class", SomeClass)
serve.create_endpoint("remote", backend="remote_class", route="/remote")

Then attempt to get_handle:

import ray
from ray import serve

ray.util.connect('ip:10001')
serve.connect()

handle = serve.get_handle('remote')

Am I doing something wrong? Thanks in advance! @rliaw @simon-mo @eoakes

2

As mentioned on slack, maybe you could try the nightly version of Ray to see if it works?

3

Nightly release works, thanks!

3 Likes