Hi folks! The Serve team is updating the 2.x API to let you run multiple Serve apps on a single Ray cluster. This will remove the limitation where you must run each Serve app on a new cluster. We weren’t aware how much this would limit some existing use cases. Our highest priority is making sure that you can continue to successfully use Serve, so we’re listening to the feedback and working to fix it.
We’ve opened an RFC detailing some of our plans, so please feel free to take a look and leave comments.
RFC: [RFC][Serve] Multi Applications in serve 2.x API · Issue #31072 · ray-project/ray · GitHub
If you feel this doesn’t satisfy your use case, please comment directly or reach out to us on Slack.