Seldon Core VS Ray Serve

Ray Libraries (Data, Train, Tune, Serve) Ray Serve
1

Hi Team
Our Team uses Kubernetes and planning to build Enterprise Inference Engine with open source frameworks. The libraries can be TF, DeepSpeed, Pytorch, Pytorch lightning, etc and machines like NVIDIA A100, V100, DGX. So I would like to know about comparison of Seldon Core VS Ray Serve.

2

Hey!

Serve is an alternative to model servers like BentoML, KFServing/KServe, Seldon Core. Serve has advanced the following differentiating features (1) fractional resource and multiplexing (2) a local development and testing workflow (3) powerful model composition capabilities.

If you are looking to scale out a single model well on K8s, you should consider Seldon Core or KServe. @simon-mo @Sihan_Wang can share more.