We have been evaluating Ray Serve and FastAPI for serving models in AWS ECS. The performance for running a single app in a single ECS node, Ray Serve lags behind FastAPI a lots (in terms of requests per second and response time).
In which use cases, Ray Serve outperforms FastAPI when serving a single or 2 models in a single application?
Thank. you