How severe does this issue affect your experience of using Ray?
- High: It blocks me to complete my task.
We’ve been looking at using the builtin autoscaling in KubeRay, however our use case is a bit more complicated as we use Ray Services which also consume from Apache Kafka (according to this post How can I integrate Apache Kafka with Ray Serve?) and as such, we want to autoscale based on different factors such as a certain lag threshold.
Therefore, we’ve been looking at KEDA to try to supplement that use case and scale accordingly, unfortunately it doesn’t seem as straightforward as we hoped it to be.
According to KEDA’s documentation on scaling CRDs, the only requirement is that the /scale subresource must be defined.
We thought that we might be able to change the RayService CRD and add the /scale subresource (according to Extend the Kubernetes API with CustomResourceDefinitions | Kubernetes), however it doesn’t seem to be possible - in RayService, we’ll need to scale the replicas of the relevant Ray Serve deployment, which its “num_replicas” field appears in serveConfigV2 - a string, which doesn’t seem to be possible to set the appropriate JSONPath in specReplicasPath.
Would appreciate help figuring this out 