FastAPI + Ray Core vs FastAPI + Ray Serve?

Great project!

I use FastAPI for the front-end of a web service, and am investigating using Ray to manage some back-end computation. Option 1 would be to use ray.serve in the documented way (create_backend, create_endpoint, get_handle, handle.remote). But there is also an Option 2 where we just call a ray.remote object directly, as in:

def hello(name:str):
  return f"Hello {name}"

async def hello_ep(name:str):
  return await hello.remote(name)

Could you outline the pros and cons of the two options please?

1 Like

Hi @Christopher_Rowley! Both ways are good. Ray Serve offer more abstractions managing, scaling, and upgrading your task (like the hello in your code sample). If you just want to simply call a Ray task to offload computation, the options 2 is totally valid. This is in fact how is using ray right now.

1 Like

Thanks. Can you say a bit more about the managing/scaling/upgrading capabilities please?