Deployments and fastapi docs

Ray Libraries (Data, Train, Tune, Serve) Ray Serve
1 
import ray

from fastapi import FastAPI
from ray import serve

app = FastAPI()
ray.init(address="auto", namespace="serve")
serve.start(detached=True)

@serve.deployment(route_prefix="/api1",
                  ray_actor_options={},
                  max_concurrent_queries=100,
                  _autoscaling_config={
                      "min_replicas": 1,
                      "max_replicas": 3,
                      "target_num_ongoing_requests_per_replica": 10},
                  version="v1")
@serve.ingress(app)
class FastAPIWrapper1:
    @app.post("/subpath")
    def method(self):
        return "Hello 1!"
    @app.post("/one")
    def one(self):
        return "the one"

@serve.deployment(route_prefix="/api2",
                  ray_actor_options={},
                  max_concurrent_queries=100,
                  _autoscaling_config={
                      "min_replicas": 1,
                      "max_replicas": 3,
                      "target_num_ongoing_requests_per_replica": 10},
                  version="v1")
@serve.ingress(app)
class FastAPIWrapper2:
    @app.post("/subpath")
    def method(self):
        return "Hello 2!"

@serve.deployment(route_prefix="/api3",
                  ray_actor_options={},
                  max_concurrent_queries=100,
                  _autoscaling_config={
                      "min_replicas": 1,
                      "max_replicas": 3,
                      "target_num_ongoing_requests_per_replica": 10},
                  version="v1" )
@serve.ingress(app)
class FastAPIWrapper3:
    @app.post("/subpath")
    def method(self):
        return "Hello 3!"

@serve.deployment(route_prefix="/api4",
                  ray_actor_options={},
                  max_concurrent_queries=100,
                  _autoscaling_config={
                      "min_replicas": 1,
                      "max_replicas": 3,
                      "target_num_ongoing_requests_per_replica": 10},
                  version="v1" )
@serve.ingress(app)
class FastAPIWrapper4:
    @app.post("/subpath")
    def method(self):
        return "Hello 4!"

FastAPIWrapper1.deploy()
FastAPIWrapper2.deploy()
FastAPIWrapper3.deploy()
FastAPIWrapper4.deploy()

When using the above code with ray 1.9.2, python 3.8.10 and fastapi 0.68.1
#ray start --head
#python3 sample.py

This results is leaking of method one from FastAPIWrapper1 to FastAPIWrapper2 instance.
This shows in http://localhost:8000/api2/docs and the “one” method is actually callable from http://localhost:8000/api2/.

Please advice on this.

2

Thanks for the detailed reproduction! I indeed reproduced this in 1.9.2, but it seems to have been fixed in the latest master build. I recommend trying a nightly wheel, or Ray 1.10.0.

3

Hello,

Thank you for the response.
My findings are 1.10.x still has this problem.
However the nighly build does not
" ray-2.0.0.dev0-cp38-cp38-manylinux2014_x86_64.whl "

4

I have a question regarding installing the .whl for nightly using pip.
I want to know if it’s possible to install a local.whl.

pip3 install --find-links=/ -U ‘ray[default] @ /ray-2.0.0.dev0-cp38-cp38-manylinux2014_x86_64.whl’

This results in a “does not exist”.

Thanks

5

Yes, it it is possible to pip install a local whl file! I’m not sure about the “does not exist” error, maybe double-check the filepath? I’m also unfamiliar with the --find-links option.