Cleanup of FastAPI Deployments

Ray Libraries (Data, Train, Tune, Serve) Ray Serve
1

Hello everyone. I’m trying to gracefully shutdown deployments due to resource allocation. Here’s the somewhat self-explanatory code snippet:

from ray import serve
from fastapi import FastAPI


serve.start()
app = FastAPI()


@serve.deployment(route_prefix="/")
@serve.ingress(app)
class MyDeployment:
    def __init__(self):
        print("I was called in init!")

    @app.on_event("startup")
    async def startup_event():
        print("I was called in startup!")

    @app.on_event("shutdown")
    async def shutdown_event():
        print("I was not called")

    def __del__(self):
        print("Neither was I")

    @app.get("/")
    async def root(self):
        return f"Hello, world!"


MyDeployment.deploy()
MyDeployment.deploy()
MyDeployment.delete()

Why neither __del__ nor shutdown events are called upon re-deployment or deletion? How do I implement cleanup routines on instances? I’m using Ray latest (1.6.0), btw.

2 Likes
2

Thanks for bringing this to our attention, @amiasato! I was able to reproduce the same locally on latest master, so this looks like a bug. I filed a GitHub issue and we will prioritize a fix.

1 Like