How severe does this issue affect your experience of using Ray?
- Low: Looking for general direction
Hi, I would like to ask for a general direction to implement a way to automate the serving of models that has been trained on the Ray Cluster.
The context is that, we have a API service that is connected to the Ray cluster, and allows user to submit their training job with custom scripts, env and datasets. After a model traing job is submitted into the Ray Cluster, the models checkpoint and related information are then stored in MinIO.
According to Ray Serve documentation, to serve a model we need to manually write a script with @serve.deployment for each model. I was wondering if there is a way we can automate the serving process without this step? Assuming we have Ray Serve deployed in a K8s cluster already.