Fastapi and response_model throws AttributeError: 'FieldInfo' object has no attribute 'init'

Ray Libraries (Data, Train, Tune, Serve) Ray Serve
1

Hey,

I’m trying to use Ray Serve with fastapi, but I’ve run into an issue while setting response_model for a route.

The code that I’m running:

from typing import List
from ray import serve

from fastapi import FastAPI
from pydantic import BaseModel

class TestModel(BaseModel):
    a: str
    b: List[str]

app = FastAPI()

@serve.deployment
@serve.ingress(app)
class TestDeployment:

    @app.get("/inner", response_model=TestModel)
    def test_endpoint_2(self):
        test_model = TestModel(a="a", b=["b"])
        return test_model

serve.run(TestDeployment.bind())

The error log that I’m getting after running the code:

2024-03-01 15:57:11,783	INFO packaging.py:530 -- Creating a file package for local directory '.'.
2024-03-01 15:57:11,794	INFO packaging.py:358 -- Pushing file package 'gcs://_ray_pkg_fdf8fc4b07c73bc4.zip' (0.21MiB) to Ray cluster...
2024-03-01 15:57:17,140	INFO packaging.py:371 -- Successfully pushed file package 'gcs://_ray_pkg_fdf8fc4b07c73bc4.zip'.
(ProxyActor pid=1006) INFO 2024-03-01 06:57:27,673 proxy 10.72.0.49 proxy.py:1143 - Proxy actor 58490891b63c9a681eb40f1604000000 starting on node c7d7c47b7daac4b97d997209124ed099db9544c12267f8d833b61a7c.
(ProxyActor pid=1006) INFO 2024-03-01 06:57:27,682 proxy 10.72.0.49 proxy.py:1357 - Starting HTTP server on node: c7d7c47b7daac4b97d997209124ed099db9544c12267f8d833b61a7c listening on port 8000
(ProxyActor pid=1006) INFO:     Started server process [1006]
(ServeController pid=972) INFO 2024-03-01 06:57:30,240 controller 972 deployment_state.py:1547 - Deploying new version of deployment TestDeployment in application 'default'. Setting initial target number of replicas to 1.
(ServeController pid=972) INFO 2024-03-01 06:57:30,343 controller 972 deployment_state.py:1831 - Adding 1 replica to deployment TestDeployment in application 'default'.
(ServeReplica:default:TestDeployment pid=293, ip=10.72.6.54) Exception raised in creation task: The actor died because of an error raised in its creation task, ray::SERVE_REPLICA::default#TestDeployment#ZxvZuZ:ServeReplica:default:TestDeployment.__init__() (pid=293, ip=10.72.6.54, actor_id=d49589738f9b1f902482dbbe04000000, repr=<ray.serve._private.replica.ServeReplica:default:TestDeployment object at 0x7f27d92192a0>)
(ServeReplica:default:TestDeployment pid=293, ip=10.72.6.54)   File "/home/ray/anaconda3/lib/python3.10/concurrent/futures/_base.py", line 458, in result
(ServeReplica:default:TestDeployment pid=293, ip=10.72.6.54)     return self.__get_result()
(ServeReplica:default:TestDeployment pid=293, ip=10.72.6.54)   File "/home/ray/anaconda3/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
(ServeReplica:default:TestDeployment pid=293, ip=10.72.6.54)     raise self._exception
(ServeReplica:default:TestDeployment pid=293, ip=10.72.6.54)   File "/home/ray/anaconda3/lib/python3.10/site-packages/ray/serve/_private/replica.py", line 108, in __init__
(ServeReplica:default:TestDeployment pid=293, ip=10.72.6.54)     deployment_def = cloudpickle.loads(serialized_deployment_def)
(ServeReplica:default:TestDeployment pid=293, ip=10.72.6.54) AttributeError: 'FieldInfo' object has no attribute 'init'

I’m using Ray 2.9.3 with Python 3.10 along with the following dependencies:

pydantic                                 2.5.0
pydantic_core                            2.14.1
fastapi                                  0.108.0

I tried to use Ray 2.7, but I’m getting the same error. I’d appreciate it if someone had an idea of how to fix it.

2

I am not sure why the above did not work but I tried a similar one as above with minor changes, notable adding the return type of test_endpoint_2 as TestModel, as in the below code and it worked:

from typing import List
from ray import serve

from fastapi import FastAPI
from pydantic import BaseModel

class TestModel(BaseModel):
    a: str
    b: List[str]

app = FastAPI()

@serve.deployment(num_replicas=1, ray_actor_options={"num_cpus": 1, "num_gpus": 0})
@serve.ingress(app)
class TestDeployment:

    @app.get("/inner", response_model=TestModel)
    def test_endpoint_2(self) -> TestModel:
        test_model = TestModel(a="a", b=["b"])
        return test_model

my_ray_app = TestDeployment.bind()

I saved the above in a file test.py and ran it as serve run test:my_ray_app. To test, visited http://localhost:8000/inner to check the response.