Hi Team
Our Team uses Kubernetes and planning to build Enterprise Inference Engine with open source frameworks. The libraries can be TF, DeepSpeed, Pytorch, Pytorch lightning, etc and machines like NVIDIA A100, V100, DGX. So I would like to know about comparison of Seldon Core VS Ray Serve.
Hey!
Serve is an alternative to model servers like BentoML, KFServing/KServe, Seldon Core. Serve has advanced the following differentiating features (1) fractional resource and multiplexing (2) a local development and testing workflow (3) powerful model composition capabilities.
If you are deploying on K8s, you should using KubeRay or Anyscale’s Kubernetes operator.