FastAPI + Ray Core vs FastAPI + Ray Serve?

Ray Serve
1

Great project!

I use FastAPI for the front-end of a web service, and am investigating using Ray to manage some back-end computation. Option 1 would be to use ray.serve in the documented way (create_backend, create_endpoint, get_handle, handle.remote). But there is also an Option 2 where we just call a ray.remote object directly, as in:

@ray.remote
def hello(name:str):
  return f"Hello {name}"

@app.get("/hello")
async def hello_ep(name:str):
  return await hello.remote(name)

Could you outline the pros and cons of the two options please?

1 Like
2

Hi @Christopher_Rowley! Both ways are good. Ray Serve offer more abstractions managing, scaling, and upgrading your task (like the hello in your code sample). If you just want to simply call a Ray task to offload computation, the options 2 is totally valid. This is in fact how anyscale.com is using ray right now.
https://youtu.be/A1dB-4FdFgQ

1 Like
3

Thanks. Can you say a bit more about the managing/scaling/upgrading capabilities please?