Multiple Serve instances on a ray cluster with serve REST API

How severe does this issue affect your experience of using Ray?

  • High: It blocks me to complete my task.

Hey guys. I’m trying to serve multiple apps with REST API but they keep replacing each other. How can I deploy multiple different serve apps with REST API or any other programmatically available method?

Hi @Ali_Sayyah , for multiple apps, do you mean multiple FastAPI apps inside the same ray serve cluster?

Currently there is no support for mutiple fast apps, but you can use mount(Sub Applications - Mounts - FastAPI) to support under the same ray serve cluster.

Hi @Sihan_Wang . Thank you for your answer. No, I don’t use FastAPI. Just a simple starlette server. Is it still impossible?
I’m trying to deploy multiple separated apps with different prefixes to the serve cluster.

It should be working by using .deploy() API, not working in Let me know if you can’t work it out, and feel free to paste reproduce script. I can take a look!

Thank you!