Unable to run FastAPI Ray Serve Deployment Example

Ray Libraries (Data, Train, Tune, Serve) Ray Serve
1

How severe does this issue affect your experience of using Ray?

  • High: It blocks me to complete my task.

I am trying to run the FastAPI HTTP Deployments:

import ray

from fastapi import FastAPI
from ray import serve

app = FastAPI()
ray.init()
serve.start(
    detached=True,
    http_options=dict(
        host="0.0.0.0",
        port=5050,
    ),
)


@serve.deployment(route_prefix="/hello")
@serve.ingress(app)
class MyFastAPIDeployment:
    @app.get("/")
    def root(self):
        return "Hello, world!"


MyFastAPIDeployment.deploy()

But I am getting the following error: ray.serve.exceptions.RayServeException: There is no instance running on this Ray cluster. Please call serve.start(detached=True) to start one. Which does not make sense as I already did start the instance.

2

@yudhiesh have you tried the docker approach? I was able to run FastAPI with Ray using docker. Reference: GitHub - dudeperf3ct/16-rayserve-deploy

3

No I have not, will definitely try that out. I found the issue, I needed to start the Ray Cluster using ray --head start before deploying the API endpoint.

4

i run docker file not work using gpus

docker run --gpus all -p 5557:1998 seo:latest , can u fix docker file