# RayServe: Failed to serialize the FastAPI app

**URL:** <https://discuss.ray.io/t/rayserve-failed-to-serialize-the-fastapi-app/22279>\
**Category:** Ray Serve\
**Created:** [April 10, 2025, 4:26pm UTC](https://discuss.ray.io/t/rayserve-failed-to-serialize-the-fastapi-app/22279 "2025-04-10T16:26:25Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Hi\_u\_Bui\_Nguy\_n\_Trun](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/hi_u_bui_nguy_n_trun/32/7837_2.png) [@Hi\_u\_Bui\_Nguy\_n\_Trun](https://discuss.ray.io/u/Hi_u_Bui_Nguy_n_Trun)\
**Post date:** [April 10, 2025, 4:26pm UTC](https://discuss.ray.io/t/rayserve-failed-to-serialize-the-fastapi-app/22279/1 "2025-04-10T16:26:25Z")

</div>

Hi eveyone,  
I try to use RayServe with FastAPI but it didn’t work. This code work well like normal FastAPI app.

```auto
app = FastAPI()

app.include_router(user_access_router, prefix='/user/access', tags=['user'])

 if __name__ ==" __main__":
     import uvicorn
     uvicorn.run("app:app",reload=True)

```

But this not :

```auto
app = FastAPI()
app.include_router(user_access_router, prefix='/user/access', tags=['user'])
@serve.deployment(name='datastore_app')
@serve.ingress(app)
class DatastoreApp:
    def __init__ (self):
        pass

 datastore_app = DatastoreApp.bind()
 serve.run(datastore_app)

```

The summary of error log like this:

```auto
TypeError: cannot pickle '_thread.lock' object
...
    @serve.ingress(app)
     ^^^^^^^^^^^^^^^^^^
TypeError: Failed to serialize the FastAPI app.
...
=======================================
!!! FAIL serialization: cannot pickle '_thread.lock' object
    Serializing 'add_api_route' <bound method FastAPI.add_api_route of <fastapi.applications.FastAPI object at 0x7bdb259e12b0>>...
    !!! FAIL serialization: cannot pickle '_thread.lock' object
        Serializing ' __func__' <function FastAPI.add_api_route at 0x7bdb210b1e40>...
    WARNING: Did not find non-serializable object in <bound method FastAPI.add_api_route of <fastapi.applications.FastAPI object at 0x7bdb259e12b0>>. This may be an oversight.
========================================

```

I inspected with:

```auto
inspect_serializability(serve.ingress(app))

```

The return:

```auto
================================================================================
Checking Serializability of <function ingress.<locals>.decorator at 0x7a16395b79c0>
================================================================================
!!! FAIL serialization: cannot pickle '_thread.lock' object
Detected 11 global variables. Checking serializability...
    Serializing 'inspect' <module 'inspect' from '/usr/lib/python3.12/inspect.py'>...
    Serializing 'collections' <module 'collections' from '/usr/lib/python3.12/collections/ __init__.py'>...
    Serializing 'Callable' typing.Callable...
    Serializing 'FastAPI' <class 'fastapi.applications.FastAPI'>...
    Serializing 'APIRouter' <class 'fastapi.routing.APIRouter'>...
    Serializing 'make_fastapi_class_based_view' <function make_fastapi_class_based_view at 0x7a163a45ad40>...
    Serializing 'ensure_serialization_context' <function ensure_serialization_context at 0x7a163c3c87c0>...
    Serializing 'cloudpickle' <module 'ray.cloudpickle' from '/home/hieu/Workspace/projects/chatbone/.venv/lib/python3.12/site-packages/ray/cloudpickle/ __init__.py'>...
    Serializing 'pickle_dumps' <function pickle_dumps at 0x7a163cb00d60>...
    Serializing 'ASGIAppReplicaWrapper' <class 'ray.serve._private.http_util.ASGIAppReplicaWrapper'>...
    Serializing ' __name__' ray.serve.api...
Detected 1 nonlocal variables. Checking serializability...
    Serializing 'app' <fastapi.applications.FastAPI object at 0x7a163f4e4920>...
    !!! FAIL serialization: cannot pickle '_thread.lock' object
        Serializing 'add_api_route' <bound method FastAPI.add_api_route of <fastapi.applications.FastAPI object at 0x7a163f4e4920>>...
        !!! FAIL serialization: cannot pickle '_thread.lock' object
            Serializing ' __func__' <function FastAPI.add_api_route at 0x7a163aab9da0>...
        WARNING: Did not find non-serializable object in <bound method FastAPI.add_api_route of <fastapi.applications.FastAPI object at 0x7a163f4e4920>>. This may be an oversight.
================================================================================
Variable: 

	FailTuple(add_api_route [obj=<bound method FastAPI.add_api_route of <fastapi.applications.FastAPI object at 0x7a163f4e4920>>, parent=<fastapi.applications.FastAPI object at 0x7a163f4e4920>])

was found to be non-serializable. There may be multiple other undetected variables that were non-serializable. 
Consider either removing the instantiation/imports of these variables or moving the instantiation into the scope of the function/class. 
================================================================================

```

I tried to serve the same endpoints fastapi without “include\_router” (using only FastAPI() and it still didn’t work. Am i missing something?

Thank you.

UPDATE:  
I traced my error and found that it happends because create sqlalchemy.AsyncSession in each FastApi endpoint (both through Dependencies or manual creation give error). But i want to do something like open session (connection), interact with database and close it in each endpoint and i still haven’t found the way to do that with Ray.

---

<div class="post-metadata">

**Author:** ![Akshay\_Malik](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/akshay_malik/32/3807_2.png) [@Akshay\_Malik](https://discuss.ray.io/u/Akshay_Malik)\
**Post date:** [April 10, 2025, 4:29pm UTC](https://discuss.ray.io/t/rayserve-failed-to-serialize-the-fastapi-app/22279/2 "2025-04-10T16:29:36Z")

</div>

You can use the route\_prefix configuration in Ray Serve applications to achieve the same functionality instead

---

<div class="post-metadata">

**Author:** ![Hi\_u\_Bui\_Nguy\_n\_Trun](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/hi_u_bui_nguy_n_trun/32/7837_2.png) [@Hi\_u\_Bui\_Nguy\_n\_Trun](https://discuss.ray.io/u/Hi_u_Bui_Nguy_n_Trun)\
**Post date:** [April 10, 2025, 4:44pm UTC](https://discuss.ray.io/t/rayserve-failed-to-serialize-the-fastapi-app/22279/3 "2025-04-10T16:44:28Z")

</div>

@Akshay_Malik Thanks for your fast response. Can you give me some words about the practice? I already have FastAPI app with some APIRouter and i need to integrate it with RayServe. How to do that? I tried to change one of APIRouter with FastAPI app (no router anymore) but ingress(app) even didn’t work.

---

<div class="post-metadata">

**Author:** ![Akshay\_Malik](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/akshay_malik/32/3807_2.png) [@Akshay\_Malik](https://discuss.ray.io/u/Akshay_Malik)\
**Post date:** [April 14, 2025, 5:29pm UTC](https://discuss.ray.io/t/rayserve-failed-to-serialize-the-fastapi-app/22279/4 "2025-04-14T17:29:38Z")

</div>

yes, APIRouter is not supported right now - [[Serve] ingress decorator does not work with fastapi.APIRouter arg · Issue #50372 · ray-project/ray · GitHub](https://github.com/ray-project/ray/issues/50372) . Can you share your code snippet? It should use the FastAPI object in ingress instead of Router

---

<div class="post-metadata">

**Author:** ![Hi\_u\_Bui\_Nguy\_n\_Trun](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/hi_u_bui_nguy_n_trun/32/7837_2.png) [@Hi\_u\_Bui\_Nguy\_n\_Trun](https://discuss.ray.io/u/Hi_u_Bui_Nguy_n_Trun)\
**Post date:** [April 17, 2025, 10:25am UTC](https://discuss.ray.io/t/rayserve-failed-to-serialize-the-fastapi-app/22279/5 "2025-04-17T10:25:03Z")

</div>

@Akshay_Malik I solved it. Actually FastApi dependency and router features work very well. I think RayServe-FastAPI integration is very good at it.

I thought the reason it fails before is that i injected the AsyncSession Dependency (Dep that yield Sqlalchemy AsyncSession) directly to endpoint parameters, and maybe Ray try to serialize the Session instead of create the object for endpoint. So i now i just yield it inside the endpoint instead of dependencies in the parameters. And it works.

Example code that run successfully:

```auto
@router.delete('/delete')
async def delete_old(schema: ChatSVCDeleteOld):
	async with get_async_session() as session:
		svc = MyService(session)
		await svc.delete_old_messages(schema)
		return dict(info="Old messages deleted successfully.")

```

---

<div class="post-metadata">

**Author:** ![Akshay\_Malik](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/akshay_malik/32/3807_2.png) [@Akshay\_Malik](https://discuss.ray.io/u/Akshay_Malik)\
**Post date:** [April 21, 2025, 4:21pm UTC](https://discuss.ray.io/t/rayserve-failed-to-serialize-the-fastapi-app/22279/6 "2025-04-21T16:21:00Z")

</div>

That’s good to hear. Can you share some code snippets of the router setup?