Unable to run FastAPI Ray Serve Deployment Example

How severe does this issue affect your experience of using Ray?

  • High: It blocks me to complete my task.

I am trying to run the FastAPI HTTP Deployments:

import ray

from fastapi import FastAPI
from ray import serve

app = FastAPI()

class MyFastAPIDeployment:
    def root(self):
        return "Hello, world!"


But I am getting the following error: ray.serve.exceptions.RayServeException: There is no instance running on this Ray cluster. Please call serve.start(detached=True) to start one. Which does not make sense as I already did start the instance.

@yudhiesh have you tried the docker approach? I was able to run FastAPI with Ray using docker. Reference: GitHub - dudeperf3ct/16-rayserve-deploy

No I have not, will definitely try that out. I found the issue, I needed to start the Ray Cluster using ray --head start before deploying the API endpoint.

i run docker file not work using gpus

docker run --gpus all -p 5557:1998 seo:latest , can u fix docker file