Ray Serve : Example not working

Ray Libraries (Data, Train, Tune, Serve) Ray Serve
1

Example mentioned at Calling Deployments via HTTP and Python — Ray v1.9.0, does not seems to be working for me.

Here is how I am running.

#main.py
import ray

from fastapi import FastAPI
from ray import serve

app = FastAPI()
ray.init(address="auto", namespace="summarizer")
serve.start(detached=True)

@app.get("/")
def f():
    return "Hello from the root!"

@serve.deployment(route_prefix="/api1")
@serve.ingress(app)
class FastAPIWrapper1:
    @app.get("/subpath")
    def method(self):
        return "Hello 1!"

@serve.deployment(route_prefix="/api2")
@serve.ingress(app)
class FastAPIWrapper2:
    @app.get("/subpath")
    def method(self):
        return "Hello 2!"

FastAPIWrapper1.deploy()
FastAPIWrapper2.deploy()

I am starting ray with ray start --head

Here is the output from post deployment

❯ python main.py
2021-12-21 13:54:55,429	INFO worker.py:842 -- Connecting to existing Ray cluster at address: 127.0.0.1:6379
(ServeController pid=94776) 2021-12-21 13:54:56,285	INFO checkpoint_path.py:16 -- Using RayInternalKVStore for controller checkpoint and recovery.
(ServeController pid=94776) 2021-12-21 13:54:56,288	INFO http_state.py:98 -- Starting HTTP proxy with name 'SERVE_CONTROLLER_ACTOR:SERVE_PROXY_ACTOR-node:127.0.0.1-0' on node 'node:127.0.0.1-0' listening on '127.0.0.1:8000'
2021-12-21 13:54:57,048	INFO api.py:463 -- Started detached Serve instance in namespace 'summarizer'.
2021-12-21 13:54:57,063	INFO api.py:242 -- Updating deployment 'FastAPIWrapper1'. component=serve deployment=FastAPIWrapper1
(HTTPProxyActor pid=94777) INFO:     Started server process [94777]
(ServeController pid=94776) 2021-12-21 13:54:57,129	INFO deployment_state.py:912 -- Adding 1 replicas to deployment 'FastAPIWrapper1'. component=serve deployment=FastAPIWrapper1
2021-12-21 13:54:57,980	INFO api.py:249 -- Deployment 'FastAPIWrapper1' is ready at `http://127.0.0.1:8000/api1`. component=serve deployment=FastAPIWrapper1
2021-12-21 13:54:57,988	INFO api.py:242 -- Updating deployment 'FastAPIWrapper2'. component=serve deployment=FastAPIWrapper2
(ServeController pid=94776) 2021-12-21 13:54:58,085	INFO deployment_state.py:912 -- Adding 1 replicas to deployment 'FastAPIWrapper2'. component=serve deployment=FastAPIWrapper2
2021-12-21 13:54:58,934	INFO api.py:249 -- Deployment 'FastAPIWrapper2' is ready at `http://127.0.0.1:8000/api2`. component=serve deployment=FastAPIWrapper2

After deployment, here is successful curl calls

❯ curl http://127.0.0.1:8000/api1/subpath
"Hello 1!"%
❯ curl http://127.0.0.1:8000/api2/subpath
"Hello 2!"%

But when I try to run curl to root, its failing.

❯ curl http://127.0.0.1:8000/
Path '/' not found. Please ping http://.../-/routes for route table.%

I am trying to assess if any endpoints that are available via FASTapi would still be available or not. Please let me know if I am missing something here.

2

Got it, path http://127.0.0.1:8000/ is not directly accessible. Due to ingress, its under the route where it is part of.

Changed path as below

@app.get("/fastapi")
def f():
    return "Hello from the root!"

Now I can call

❯ curl http://127.0.0.1:8000/api1/fastapi
"Hello from the root!"