Running multiple FastAPIs with serve

Ray Core
1

How severe does this issue affect your experience of using Ray?

  • High: It blocks me to complete my task.

I have multiple FastAPIs that I would like to serve on a single cluster, but from what I can tell there is only the option to deploy one, as you are limited to a single serve.ingress() - HTTP Handling — Ray 2.1.0

My goal is to be able to easily import and use several FastAPIs, like you can do for one with serve.ingress(app), rather than having to copy in all the individual endpoints into one big messy file. Are there any ways to do this?

2

Hi @htess , currently serve only support one FastAPI app, but you can have other ways to achiieve it (if only concern is big messy file)

  1. subapp mount: Sub Applications - Mounts - FastAPI
  2. One app with multiple class: ray/test_fastapi.py at master · ray-project/ray · GitHub

Thank you!