FastAPI and GraphQL

I’m trying to get Ray Serve to handle a FastAPI endpoint with GraphQL but getting what seems to be some serialization error.

From reading GraphQL - FastAPI and Calling Deployments via HTTP and Python — Ray v1.8.0 i got to the following snippet.

Dependencies:

ray[default]==1.8.0
ray[serve]==1.8.0
aiohttp>=3.7, <3.8 # ray has >3.7 but 3.8 breaks it

fastapi==0.70.0
strawberry-graphql==0.90.2

Code:

import ray
import strawberry
from fastapi import FastAPI
from ray import serve
from strawberry.asgi import GraphQL


@strawberry.type
class User:
    name: str
    age: int


@strawberry.type
class Query:
    @strawberry.field
    def user(self) -> User:
        return User(name="Patrick", age=100)


schema = strawberry.Schema(query=Query)
graphql_app = GraphQL(schema)

app = FastAPI()
app.add_route("/graphql", graphql_app)


ray.init(address="auto", namespace="graphql")
serve.start(detached=True)


@serve.deployment(route_prefix="/")
@serve.ingress(app)
class FastAPIWrapper:
    pass


FastAPIWrapper.deploy()

This is throwing me this error when executing:

2021-11-30 17:43:26,556 INFO worker.py:822 -- Connecting to existing Ray cluster at address: 127.0.0.1:6379
2021-11-30 17:43:27,435 INFO checkpoint_path.py:15 -- Using RayInternalKVStore for controller checkpoint and recovery.
2021-11-30 17:43:27,438 INFO http_state.py:98 -- Starting HTTP proxy with name 'SERVE_CONTROLLER_ACTOR:SERVE_PROXY_ACTOR-node:127.0.0.1-0' on node 'node:127.0.0.1-0' listening on '127.0.0.1:8000'
2021-11-30 17:43:28,229 INFO api.py:441 -- Started detached Serve instance in namespace 'graphql'.
Traceback (most recent call last):
  File "serve.py", line 34, in <module>
    class FastAPIWrapper:
  File "/Users/hfernandes/Projects/ray-serve/.venv/lib/python3.8/site-packages/ray/serve/api.py", line 554, in decorator
    frozen_app = cloudpickle.loads(cloudpickle.dumps(app))
  File "/Users/hfernandes/Projects/ray-serve/.venv/lib/python3.8/site-packages/graphql/pyutils/frozen_list.py", line 48, in extend
    raise FrozenError
graphql.pyutils.frozen_error.FrozenError

Am i doing something wrong?

hi @hbfernandes, looks like serve didn’t support serialization for this case and i’ve filed an issue for it in [Feature] Support strawberry-graphql in serve with FastAPI · Issue #20805 · ray-project/ray · GitHub (feel free to suggest more changes if you want).

As a walkaround i think what i found at the bottom of GraphQL - FastAPI might help that uses starlette-graphene3 instead since we support Starlette in serve app serialization. Do you mind giving it try to it and let us know if you run into more issues ?

Hey @jiaodong

I’ve tried with the following snippet for graphene 2.1.9.

import ray
from fastapi import FastAPI
from graphene import ObjectType, Schema, String
from ray import serve
from starlette.graphql import GraphQLApp


class Query(ObjectType):
    # this defines a Field `hello` in our Schema with a single Argument `name`
    hello = String(name=String(default_value="stranger"))
    goodbye = String()

    # our Resolver method takes the GraphQL context (root, info) as well as
    # Argument (name) for the Field and returns data for the query Response
    def resolve_hello(root, info, name):
        return f'Hello {name}!'

    def resolve_goodbye(root, info):
        return 'See ya!'

schema = Schema(query=Query)

graphql_app = GraphQLApp(schema)

app = FastAPI()
app.add_route("/graphql", graphql_app)


ray.init(address="auto", namespace="graphql")
serve.start(detached=True)


@serve.deployment(route_prefix="/")
@serve.ingress(app)
class FastAPIWrapper:
    pass


FastAPIWrapper.deploy()

And got the error:

2021-12-02 10:59:32,674 INFO worker.py:822 -- Connecting to existing Ray cluster at address: 127.0.0.1:6379
2021-12-02 10:59:33,599 INFO checkpoint_path.py:15 -- Using RayInternalKVStore for controller checkpoint and recovery.
2021-12-02 10:59:33,603 INFO http_state.py:98 -- Starting HTTP proxy with name 'SERVE_CONTROLLER_ACTOR:SERVE_PROXY_ACTOR-node:127.0.0.1-0' on node 'node:127.0.0.1-0' listening on '127.0.0.1:8000'
INFO: Started server process [49544]
2021-12-02 10:59:34,362 INFO api.py:441 -- Started detached Serve instance in namespace 'graphql'.
Traceback (most recent call last):
  File "serve-phene.py", line 35, in <module>
    class FastAPIWrapper:
  File "/Users/hfernandes/Projects/ray-serve/.venv/lib/python3.8/site-packages/ray/serve/api.py", line 554, in decorator
    frozen_app = cloudpickle.loads(cloudpickle.dumps(app))
TypeError: __init__() missing 1 required positional argument: 'types'

Then swapped to graphene 3.0 and starlette-graphene3==0.5.1 and replaced the GraphQLApp import to

from starlette_graphene3 import GraphQLApp

and the error was the same as with Strawberry.

I feel that the problem is with serializing something on the graphql-core library, which both graphene and strawberry depend uppon.

Thanks for trying out on this, I think you’ve uncovered a pattern that other uses might run into as well about serialization. We force serializing fastapi app but it seems like blocking use cases like yours.

The good news is, it’s possible to avoid serializing it to get rid of this class of failure. @simon-mo is working on a one pager for this.

1 Like

@hbfernandes - were you able to get this working?

@jiaodong - is the one-pager available?

@miked No sorry, haven’t tried it again since i posted it. The ticket is still open so i guess the problem should remain.