Ray serve: no attribute 'add_done_callback'

Ray Libraries (Data, Train, Tune, Serve) Ray Serve
1

I try the basic HTTP ingress example of Ray Serve (Key Concepts — Ray 2.8.0), but whenever I launch a post request, I get a AttributeError: ‘ray._raylet.ObjectRef’ object has no attribute ‘add_done_callback’ error (complete stack trace below). This happens in the FastAPI ingress example.

In the raw starlette Request example, I get the error:

TypeError: ‘coroutine’ object is not subscriptable

On this line:

name = await request.json()[“name”]

I use a poetry project in which I have installed the following:

[tool.poetry.dependencies]
python = “^3.11”
flask = “^2.3.2”
pyopenssl = “^23.1.1”
gunicorn = “^20.1.0”
ray = {extras = [“serve”], version = “^2.4.0”}
requests = “^2.30.0”
fastapi = “^0.95.2”

The Ray server is launched using the command

poetry run serve run --port 4444 --host 0.0.0.0 src.project.model.serve:basic_ingress

Complete stack trace:

(HTTPProxyActor pid=1781699) ERROR: Exception in ASGI application
(HTTPProxyActor pid=1781699) Traceback (most recent call last):
(HTTPProxyActor pid=1781699) File “/.venv/lib/python3.11/site-packages/uvicorn/protocols/http/h11_impl.py”, line 428, in run_asgi
(HTTPProxyActor pid=1781699) result = await app( # type: ignore[func-returns-value]
(HTTPProxyActor pid=1781699) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(HTTPProxyActor pid=1781699) File “/.venv/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py”, line 78, in call
(HTTPProxyActor pid=1781699) return await self.app(scope, receive, send)
(HTTPProxyActor pid=1781699) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(HTTPProxyActor pid=1781699) File “/.venv/lib/python3.11/site-packages/ray/serve/_private/http_proxy.py”, line 404, in call
(HTTPProxyActor pid=1781699) status_code = await _send_request_to_handle(handle, scope, receive, send)
(HTTPProxyActor pid=1781699) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(HTTPProxyActor pid=1781699) File “/.venv/lib/python3.11/site-packages/ray/serve/_private/http_proxy.py”, line 117, in _send_request_to_handle
(HTTPProxyActor pid=1781699) _, request_timed_out = await asyncio.wait(
(HTTPProxyActor pid=1781699) ^^^^^^^^^^^^^^^^^^^
(HTTPProxyActor pid=1781699) File “/.conda/envs/python3.11/lib/python3.11/asyncio/tasks.py”, line 418, in wait
(HTTPProxyActor pid=1781699) return await _wait(fs, timeout, return_when, loop)
(HTTPProxyActor pid=1781699) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(HTTPProxyActor pid=1781699) File “/.conda/envs/python3.11/lib/python3.11/asyncio/tasks.py”, line 522, in _wait
(HTTPProxyActor pid=1781699) f.add_done_callback(_on_completion)
(HTTPProxyActor pid=1781699) ^^^^^^^^^^^^^^^^^^^
(HTTPProxyActor pid=1781699) AttributeError: ‘ray._raylet.ObjectRef’ object has no attribute ‘add_done_callback’

2

FYI: I solved it by using Python 3.10 instead of Python 3.11…

1 Like
3

Thank you for posting a solution to your problem as well.
Is there maybe a potential solution which supports use of Python 3.11 as well? :slight_smile:

1 Like
4

I see a very similar issue after upgrading to python 3.11.

  • ray: 2.38.0
  • fastapi: 0.94.1
AttributeError: 'DeploymentResponse' object has no attribute 'add_done_callback'

I’m looking more into FastAPI – maybe it’s solved in later versions?

5

Hi @Oblynx could you post more info on what code you ran, and where you saw this error?

6

Hey, coming back to this with some extra info.

The object is ray.serve.handle.DeploymentResponse. It ends up raising:

>           f.add_done_callback(_on_completion)
E           AttributeError: 'DeploymentResponse' object has no attribute 'add_done_callback'

/usr/lib/python3.11/asyncio/tasks.py:532: AttributeError

Why are we trying to use it this way?
The user code does:

  • await asyncio.wait([coro], timeout=timeout)
  • where coro: ray.serve.handle.DeploymentResponse = model_handle.preprocess.remote(...)
  • where model_handle: ray.serve.handle.DeploymentHandle
7

Essentially what happens in Python 3.11 is that asyncio.wait calls the method add_done_callback that is defined in the stdlib Task and Future interface.

I currently use this workaround:

if isinstance(coro, DeploymentResponse):
    coro = asyncio.ensure_future(coro)
await asyncio.wait([coro], timeout=timeout)