Ray and Rapids cuDF

Ray Core
1

Hi,
I was wondering how I could and if it makes sense to use Ray and Rapids cuDF together. The alternatives in my mind are(and a newbie to Ray):

  • Is there something like the dask-cudf integration (Multi-GPU with Dask-cuDF — cudf 0.15.0 documentation) for ray and cudf and if that makes sense?
  • Enable GPU support in Ray [1]. Just use Mars/Modin(sorry, I don’t know the difference) on top of it. That should be equivalent to dask-cudf?
  • There is a Dask on Ray :). Can I use dask-cudf while running Dask on Ray?
  • Use cuDF within ray.remote functions

Do you know of any benchmarks for these or initial guesses on which would perform faster?

[1] - GPU Support — Ray v2.0.0.dev0

2

Hey @devin-petersohn, do you know what 's the current status of Modin + CuDF?

3

Also @Clark_Zinzow can we use the cuDF with dask on ray? My impression is there’s no reason this won’t work unless dask distributed is doing something special

4

It’s coming along. We have excellent early performance numbers. It’s not in the main branch of Modin yet because it’s quite early, but if anyone is interested in trying it I can point them in the right direction. It’s still open source, just needs to be installed differently.

2 Likes
5

@devin-petersohn We would be very interested. However, we don’t need it now. We can wait. Is there a particular future version that you see this functionality in? or in other words, where do we go to find out that it’s in Ray (e.g release notes, mailing list)

6

You can follow Modin release notes: Releases · modin-project/modin · GitHub

Modin releases independently from Ray.

7

How is Modin + cuDF coming along ?