Ray Serve, FastAPI python package/module example?

Ray Libraries (Data, Train, Tune, Serve) Ray Serve
1

Disclosure: Every other language developer, new to python

In looking at the ray serve examples and some of the blogs, it seems “everything” is globbed into one file (the fastapi, the ray init, the ray served “model” that wraps some other ML model like Resnet or whatever. As I’m building out a proof of concept app to show ray’s capabilities, I’m trying to build it in a organized, separation of responsibility way, but in the python “way”. To that end, I have an “api” package, “classify” package, and then a ray package, where the main app uses the api module, which uses the classify module which uses the ray module. Well that’s the plan. Just wondering if anyone has built or an example of a “modularized” ray serve app and or has any recommendations? If that is too vague of a question or if a sample of how I’m layout the code (conceptually right now) , I can put up a repo

2

One way to separate out the responsibilities is to have one script for defining the model code and deploying Ray Serve, and a second script that has the main app with the HTTP routes, like in this example: Integration with Existing Web Servers — Ray v2.0.0.dev0.

1 Like
3

Thanks, I’ve also come across this which has given me a few ideas wrt how things get configured and setup in a “bigger” application

1 Like