Cloudflow(RISELab) and Ray Serve Deployment Graph API

Ray Libraries (Data, Train, Tune, Serve) Ray Serve
1

Hi,

I’m really enjoying reading “Optimizing Prediction Serving on Low-Latency Serverless Dataflow” by Vikram Sreekanti et al. Paper:https://arxiv.org/pdf/2007.05832.pdf.

The Dataflow API (map, filter, aggregate over a Table datastructure) mentioned in the paper looks very interesting.

The more I read about the optimizations(fusing, co-location), the more it reminds me of the Ray Serve Deployment Graph API.

Questions:

  • What is Ray Serve team’s opinion about such a dataflow API over the Deployment Graph API?
  • I’m wondering if the Ray Serve team is looking into something like this and if it’s in the roadmap somewhere.
  • Given the academic background of a lot of the Ray Serve team, are there other papers or ideas that the team is looking at instead. I’m just curious about what you are reading :slight_smile:

Regards,
Rajiv

2

Hi @rabraham thanks for the question :slight_smile: we are definitely aware of the Dataflow work and had similar motivations for building the deployment graph API – one of the primary reasons for it is to enable us to do more optimizations under the hood such as fusing, co-locating, and competitive execution. As of now we don’t have plans to implement these immediately, but we have them in mind to make sure the deployment graph API leaves us open to automatic optimizations as much as possible.

cc @simon-mo if you have more thoughts or other pointers!

1 Like