# New FastAPI HTTP Deployments running on uvicorn

**URL:** <https://discuss.ray.io/t/new-fastapi-http-deployments-running-on-uvicorn/3435>\
**Category:** Ray Serve\
**Created:** [September 2, 2021, 2:44pm UTC](https://discuss.ray.io/t/new-fastapi-http-deployments-running-on-uvicorn/3435 "2021-09-02T14:44:32Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Javier\_Bosch](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/javier_bosch/32/150_2.png) [@Javier\_Bosch](https://discuss.ray.io/u/Javier_Bosch)\
**Post date:** [September 2, 2021, 2:44pm UTC](https://discuss.ray.io/t/new-fastapi-http-deployments-running-on-uvicorn/3435/1 "2021-09-02T14:44:32Z")

</div>

I am a bit confused by the new FastAPI HTTP Deployment and I am having trouble getting it running with my existing application.

I am currently running my application with uvicorn on port `8080`. I am able to launch serve with `http_options=dict(location='NoServer")`

I tried using APIRouter and the app instance, but I keep on getting an error

`ray.serve.exceptions.RayServeException: `serve.get\_replica\_context()` may only be called from within a Ray Serve backend.`

serve is started as follows:

```auto
ray.init(address="auto", ignore_reinit_error=True, namespace="serve")
serve.start(
            http_options=dict(location="NoServer"), detached=True,
        )

```

FastAPI is started as follows:

```auto
ray.init(address="auto", ignore_reinit_error=True, namespace="serve")

uvicorn.run(
        "app:app", port=project_config.app.port, log_level=project_config.app.log_level
    )

```

My code is taken from the example:

```auto
from fastapi import FastAPI
from ray import serve

app = FastAPI()

@serve.deployment(name="deployment", route_prefix="/api")
@serve.ingress(app)
class MyFastAPIDeployment:
    @app.get("/")
    def root_handler(self):
        return "hello from /api"

    @app.get("/{subpath}")
    def subpath_handler(self, subpath: str):
        return f"hello from /{subpath}"

MyFastAPIDeployment.deploy()

```

---

<div class="post-metadata">

**Author:** ![amiasato](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/amiasato/32/1409_2.png) [@amiasato](https://discuss.ray.io/u/amiasato)\
**Post date:** [September 2, 2021, 5:02pm UTC](https://discuss.ray.io/t/new-fastapi-http-deployments-running-on-uvicorn/3435/2 "2021-09-02T17:02:15Z")

</div>

Afaik, you shouldn’t run uvicorn directly in the serve runtime, since the cluster is responsible for maitaining [a single uvicorn server per node](https://docs.ray.io/en/latest/serve/architecture.html). What you should do instead is call `serve.start()` to initialize the said runtime prior to deployment, or follow the [deployment guide](https://docs.ray.io/en/latest/serve/deployment.html#) to serve in a long-lived Ray cluster.

---

<div class="post-metadata">

**Author:** ![Javier\_Bosch](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/javier_bosch/32/150_2.png) [@Javier\_Bosch](https://discuss.ray.io/u/Javier_Bosch)\
**Post date:** [September 2, 2021, 5:32pm UTC](https://discuss.ray.io/t/new-fastapi-http-deployments-running-on-uvicorn/3435/3 "2021-09-02T17:32:05Z")

</div>

Hi @amiasato ,

I launched ray serve in detached mode and without an http server. In my case I am using my own uvicorn server.

---

<div class="post-metadata">

**Author:** ![amiasato](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/amiasato/32/1409_2.png) [@amiasato](https://discuss.ray.io/u/amiasato)\
**Post date:** [September 3, 2021, 6:18pm UTC](https://discuss.ray.io/t/new-fastapi-http-deployments-running-on-uvicorn/3435/4 "2021-09-03T18:18:53Z")

</div>

Do you have any reason to use you own uvicorn server? This was one of [the old ways](https://docs.ray.io/en/latest/serve/tutorials/web-server-integration.html?highlight=FastAPI#scaling-up-a-fastapi-application) of integrating FastAPI with Ray Serve and is currently deprecated.

---

<div class="post-metadata">

**Author:** ![eoakes](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/eoakes/32/7825_2.png) [@eoakes](https://discuss.ray.io/u/eoakes)\
**Post date:** [September 3, 2021, 11:02pm UTC](https://discuss.ray.io/t/new-fastapi-http-deployments-running-on-uvicorn/3435/5 "2021-09-03T23:02:19Z")

</div>

@Javier_Bosch I think there is a bit of a mixture of approaches happening here. In general, as @amiasato mentioned the **recommended way to integrate with FastAPI is the [native integration](https://docs.ray.io/en/latest/serve/http-servehandle.html#fastapi-http-deployments)**. If you want to run your own HTTP server and call Serve from within it, you can do that but you don’t need to use the `@serve.ingress` decorator at all. You can simply deploy and call your deployment using the regular Python API:

```auto
# This can all happen in your FastAPI server.
Deployment.deploy()
handle = Deployment.get_handle()
ray.get(handle.remote())

```