Problem with FastAPI's Background Tasks

Ray Serve
6

Hey guys! I tried to add background tasks to my Ray Serve endpoints, but I followed the official docs of FastAPI and found BackgroundTasks from FastAPI cannot achieve my goal as the add_tasks method is not an async function.

After a bit of workarounds in this topic, the correct answer is to directly use the BackgroundTask or BackgroundTasks from the starlette.background module. Generally, the boilerplate will be like -

import uuid

import ray
from ray.serve.handle import DeploymentHandle
from starlette.background import BackgroundTask
from starlette.responses import JSONResponse

app = FastAPI()

@serve.deployment
@serve.ingress(app)
class YourAppIngress:
    def __init__(self, deployment_handler: DeploymentHandle):
        self.handler = deployment_handler
    
    @app.post("/schedule")
    def schedule(self, path_or_query_or_body_params_as_usual: str):
        try:
            # Optional session_id, or task_id, etc.
            session_id = str(uuid.uuid1())
            your_background_task = BackgroundTask(
                self.handler.handle_long_running_request.remote,
                session_id,
                path_or_query_or_body_params_as_usual,
                other_params_that_you_need_to_pass_to_your_task_function
            )
        except Exception as e:
            raise HTTPException(status_code=404, detail=f"Failed to schedule due to: {e}") from None
        return JSONResponse(
            content={
                "session_id": session_id,
                "message": "Successfully scheduled.",
            },
            background= your_background_task,
        )

:slightly_smiling_face: Hope it would help those who are struggling to find a solution.

1 Like