# Ray cluster shutting down while fastapi app stays alive

**URL:** <https://discuss.ray.io/t/ray-cluster-shutting-down-while-fastapi-app-stays-alive/1332>\
**Category:** Ray Core\
**Created:** [March 19, 2021, 10:16pm UTC](https://discuss.ray.io/t/ray-cluster-shutting-down-while-fastapi-app-stays-alive/1332 "2021-03-19T22:16:12Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![delioda79](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/delioda79/32/312_2.png) [@delioda79](https://discuss.ray.io/u/delioda79)\
**Post date:** [March 19, 2021, 10:16pm UTC](https://discuss.ray.io/t/ray-cluster-shutting-down-while-fastapi-app-stays-alive/1332/1 "2021-03-19T22:16:12Z")

</div>

I’ve got a service running with FastAPI and connecting to a ray cluster. My issue is that when the ray cluster crashes, for some reason, the fastAPI application stays alive, making the whole service unresponsive. I’d need a way to stop the whole application, letting crash the pod. AT the moment I cannot even get notified of the situation if I do not look at the logs

---

<div class="post-metadata">

**Author:** ![delioda79](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/delioda79/32/312_2.png) [@delioda79](https://discuss.ray.io/u/delioda79)\
**Post date:** [March 20, 2021, 8:00am UTC](https://discuss.ray.io/t/ray-cluster-shutting-down-while-fastapi-app-stays-alive/1332/2 "2021-03-20T08:00:19Z")

</div>

SOme mor edetails:

I have one pod starting a ray cluster with  
ray tart --head  
And in the same pod I then star a fastapi app which I serve with gunicorn and uvicorn workers.  
The app creates one ray actor which is essentially a pika client which consumes from a specific queue and does heavy calculation (it is a preprocessing/training ML application). WHht at some point happens is that at some point the ray cluster stops, not sure why, trying to find out in the logs but no luck yet. Now my big issue is not that the ray cluster stops, it might just go out of memory and it is fine, but the problem is that when the ray cluster stops the gunicorn served application does not stop, it keeps restarting the workers, which will fail forever as the ray cluster instance is off. Now I’d need to have a way to stop the gunicorn app if the ray client gets disconnected but I cannot find it.

---

<div class="post-metadata">

**Author:** ![delioda79](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/delioda79/32/312_2.png) [@delioda79](https://discuss.ray.io/u/delioda79)\
**Post date:** [March 20, 2021, 8:02am UTC](https://discuss.ray.io/t/ray-cluster-shutting-down-while-fastapi-app-stays-alive/1332/3 "2021-03-20T08:02:51Z")

</div>

import ray  
from fastapi import FastAPI  
from domain.dummy\_worker import DummyWorker

app = FastAPI()

@app.get("/")  
def read\_root():  
return {“Hello”: f"World {ray.get(dummy2.gimme\_a\_number.remote())}"}

@app.on\_event(“startup”)  
def main():  
ray.init(address=“auto”, dashboard\_host=“127.0.0.1”,  
dashboard\_port=8260,  
\_memory=78643200)

```
Dummy = ray.remote(DummyWorker)

dummy = Dummy.remote()

dummy.run.remote()

global dummy2
dummy2 = Dummy.remote()

```

if **name** == “ **main** ”:  
main()

---

<div class="post-metadata">

**Author:** ![delioda79](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/delioda79/32/312_2.png) [@delioda79](https://discuss.ray.io/u/delioda79)\
**Post date:** [March 20, 2021, 8:06am UTC](https://discuss.ray.io/t/ray-cluster-shutting-down-while-fastapi-app-stays-alive/1332/4 "2021-03-20T08:06:54Z")

</div>

in order to run it I have an sh script which I run in my docker container as CMD entry point,  
the sh script essentually does:

```auto
ray start --head

sh start.sh

```

and start.sh is essentially the one available in tinagolo’s docker image, which configures gunicorn and at the end just does:

```auto
gunicorn -k blabla.UvicornWOrker $"APPLICATION_NAME"

```

Now when ray crashes for some reason, the application becomes unresponsive

---

<div class="post-metadata">

**Author:** ![sangcho](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/sangcho/32/425_2.png) [@sangcho](https://discuss.ray.io/u/sangcho)\
**Post date:** [March 25, 2021, 8:51am UTC](https://discuss.ray.io/t/ray-cluster-shutting-down-while-fastapi-app-stays-alive/1332/5 "2021-03-25T08:51:34Z")

</div>

So, if there’s a way to detect Ray failure, that will be sufficient to solve your problem?

---

<div class="post-metadata">

**Author:** ![sangcho](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/sangcho/32/425_2.png) [@sangcho](https://discuss.ray.io/u/sangcho)\
**Post date:** [March 25, 2021, 8:54am UTC](https://discuss.ray.io/t/ray-cluster-shutting-down-while-fastapi-app-stays-alive/1332/6 "2021-03-25T08:54:15Z")

</div>

e.g., you can have a background task that checks the health of ray, and when it detects Ray failures, it can kill the fast API process.