Set max_num_models_per_replica for multiplexed deployments in the serve config file

Ray Libraries (Data, Train, Tune, Serve) Ray Serve
1

How severe does this issue affect your experience of using Ray?

  • High: It blocks me to complete my task.

Is it possible to set the max_num_models_per_replica for multiplexed deployments in the serve config file, so that the application can be updated in place?

1 Like
2

Could you submit a feature request on GitHub, so we can track this?

3

Done! [Serve] Make `max_num_models_per_replica` in `@serve.multiplexed` reconfigurable · Issue #46422 · ray-project/ray · GitHub