Non ml ray uses

Ray Core
1

Ray seems built for ML but do you think it would be a good fit also for any other project, even say a basic web API, where I could scale up the requests so that each request would have an actor or a remote function producing the result?

2

Great question!

You can certainly build web apps, for example, see

Here are a good talk on how Ant Financial uses Ray to build a fusion engine performing a wide range of compute tasks (e.g., streaming, graph, batch, ML, etc workloads)

Some production use cases involving ML-based applications.

Note that many of these applications involve ML in some form, often integrated into broader application logic.

3

Well, apart from the ML projects we have were I work, we also have set of microservices, so I was wondering if using ray for our microservices could be a good thing, in order to make good use of CPUs when we run our microservices. Thing is I come from using mainly Go, and there I can spin as many go routines as I want, I was thinking if with ray I could achieve something similar, where the remote functions or actors could make use of parallelism without me having to care of proccesses by myself. The only problem, form what I see,is that I should deploy each microservice in a new ray cluster, not sure if this is a good thing for something with potentially 100s of microservices

4

I mean I could also have ray running on a single pod per service instance, but I was after achieving something similar to what akka does

5

It’s hard to say without knowing more about your use case, but it sounds like you may want to deploy all the microservices in a single Ray cluster.

6

So you confirm that ML is just an application of distributed workload for Ray and that other distributed workload could be built using Ray Serve ?

7

That’s right. You can build very sophisticated non ML applications on top of Ray! Take a look at some of the talks here from the last Ray Summit. https://www.youtube.com/playlist?list=PLzTswPQNepXlr2L75tZt7l8tyVZkO-RVt

8

I’ve actually done some experiments and for now I don’t have, to be honest, profiled any increase in performance. But this might depend on multiple things. It is a FastAPI app, like the one in the talks you posted, and I tried to use Ray to distribute the repositories (DDD data layer) as different ray Actors. Each actor would be part of a Pool (handmade) and every time a route needs to get something from the DB the pool will contact an actor of that specific repository (i.e. Cart repo) to get the data back. Well I run ray on one single pod on K8S (staging env) and FastAPI runs with default tiangolo docker settings, which is 2 uvicorn workers per CPU. This means that it could only work in concurrency, as already the python processes are twice as the number of CPUs, and the ACtors’ processes will run in the same CPUs. Now this project is very simple, and I believe this I said, plus the fact that variables are pickled, might add actually a lag and decrease the performance. Now this does not mean ray is not a good fit for non ML, but the application should be first of all worth it, not a simple CRUD service, and then resources specifically allocated. In a nutshell, just adding a bit of ray into an existing app would give, in my opinion and the short experience till now, no benefits and maybe even worst performance