Ray serve example does not work

Ray Libraries (Data, Train, Tune, Serve) Ray Serve
1

Hi,

I ran the example code from here

import ray
from ray import serve
import requests

ray.init(num_cpus=4)
client = serve.start()


def say_hello(request):
    return "hello " + request.query_params["name"] + "!"


# Form a backend from our function and connect it to an endpoint.
client.create_backend("my_backend", say_hello)
client.create_endpoint("my_endpoint", backend="my_backend", route="/hello")

# Query our endpoint in two different ways: from HTTP and from Python.
print(requests.get("http://127.0.0.1:8000/hello?name=serve").text)
# > hello serve!
print(ray.get(client.get_handle("my_endpoint").remote(name="serve")))
# > hello serve!

Here is the output:

(base) ray@1214b066f43e:/home/ray_server$ python main.py
2021-02-08 07:07:51,247 INFO services.py:1173 -- View the Ray dashboard at http://127.0.0.1:8265
2021-02-08 07:07:51,252 WARNING services.py:1640 -- WARNING: The object store is using /tmp instead of /dev/shm because /dev/shm has only 67108864 bytes available. This may slow down performance! You may be able to free up space by deleting files in /dev/shm or terminating any running plasma_store_server processes. If you are inside a Docker container, you may need to pass an argument with the flag '--shm-size' to 'docker run'.
(pid=2033) 2021-02-08 07:07:52,046      INFO controller.py:346 -- Starting router with name 'qbGXtl:SERVE_CONTROLLER_ACTOR:SERVE_PROXY_ACTOR-node:172.17.0.2-0' on node 'node:172.17.0.2-0' listening on '127.0.0.1:8000'
(pid=2032) INFO:     Started server process [2032]
(pid=2033) 2021-02-08 07:07:54,453      INFO controller.py:753 -- Registering route '/hello' to endpoint 'my_endpoint' with methods '['GET']'.
Internal Server Error
2021-02-08 07:07:54,510 INFO router.py:224 -- Endpoint my_endpoint doesn't exist, waiting for registration.
Traceback (most recent call last):
  File "main.py", line 24, in <module>
    print(ray.get(client.get_handle("my_endpoint").remote(name="serve")))
  File "/home/ray/anaconda3/lib/python3.7/site-packages/ray/worker.py", line 1379, in get
    raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(AttributeError): ray::RayServeReplica_say_hello.handle_request() (pid=2142, ip=172.17.0.2)
  File "/home/ray/anaconda3/lib/python3.7/site-packages/ray/serve/backend_worker.py", line 136, in wrap_to_ray_error
    raise exception
  File "/home/ray/anaconda3/lib/python3.7/site-packages/ray/serve/backend_worker.py", line 256, in invoke_single
    result = await method_to_call(arg)
  File "/home/ray/anaconda3/lib/python3.7/site-packages/ray/async_compat.py", line 29, in wrapper
    return func(*args, **kwargs)
  File "main.py", line 14, in say_hello
    return "hello " + request.query_params["name"] + "!"
AttributeError: 'ServeRequest' object has no attribute 'query_params'
(pid=2032) ERROR:    Exception in ASGI application
(pid=2032) Traceback (most recent call last):
(pid=2032)   File "/home/ray/anaconda3/lib/python3.7/site-packages/uvicorn/protocols/http/h11_impl.py", line 394, in run_asgi
(pid=2032)     result = await app(self.scope, self.receive, self.send)
(pid=2032)   File "/home/ray/anaconda3/lib/python3.7/site-packages/uvicorn/middleware/proxy_headers.py", line 45, in __call__
(pid=2032)     return await self.app(scope, receive, send)
(pid=2032)   File "/home/ray/anaconda3/lib/python3.7/site-packages/ray/serve/http_proxy.py", line 124, in __call__
(pid=2032)     result = await ref
(pid=2032) ray.exceptions.RayTaskError(AttributeError): ray::RayServeReplica_say_hello.handle_request() (pid=2142, ip=172.17.0.2)
(pid=2032)   File "/home/ray/anaconda3/lib/python3.7/site-packages/ray/serve/backend_worker.py", line 136, in wrap_to_ray_error
(pid=2032)     raise exception
(pid=2032)   File "/home/ray/anaconda3/lib/python3.7/site-packages/ray/serve/backend_worker.py", line 256, in invoke_single
(pid=2032)     result = await method_to_call(arg)
(pid=2032)   File "/home/ray/anaconda3/lib/python3.7/site-packages/ray/async_compat.py", line 29, in wrapper
(pid=2032)     return func(*args, **kwargs)
(pid=2032)   File "main.py", line 14, in say_hello
(pid=2032)     return "hello " + request.query_params["name"] + "!"
(pid=2032) AttributeError: 'Request' object has no attribute 'query_params'
2

Hi @tim,

Sorry you ran into this! I think the problem is that you are using the documentation for the master branch, but you aren’t running the Ray nightly release. You can either switch to the Ray nightly release, or continue to use the latest pip release alongside the docs for the latest pip release here: Ray Serve: Scalable and Programmable Serving — Ray v1.1.0

We’ll make the version differences more prominent in the documentation.

1 Like
3

Hi,

thank you very much for your help,

I will try that !

4

Thank you very much it works now. I am using a docker container so it is working now, but on my local machine (outside from the container) I have a new error:

AttributeError: Can't get attribute '_Pydantic_BackendMetadata_94776357486936' o             n <module 'pydantic.dataclasses' from 'somepath/lib/python3.6/site-packages/pydantic/dataclasses.cpython-36m-x86_6             4-linux-gnu.so'>

It is raised by the following code:

client.create_backend("hello", echo)

The entire code is here:

from ray import serve
import requests

client = serve.start()

def echo(flask_request):
    return "hello " + flask_request.args.get("name", "serve!")

client.create_backend("hello", echo)
client.create_endpoint("hello", backend="hello", route="/hello")
print(requests.get("http://127.0.0.1:8000/hello").text)

Do you have an idea on how to solve this error ?

5

Hi @tim, the issue is probably related to this issue: PicklingError in ray serve · Issue #11648 · ray-project/ray · GitHub

Can you try running the command pip install -U pydantic==1.6 on your local machine to see if that fixes it?

1 Like
6

It solves my problem, than kyou very much for your help :slight_smile: