Ray and Rapids cuDF

Hi,
I was wondering how I could and if it makes sense to use Ray and Rapids cuDF together. The alternatives in my mind are(and a newbie to Ray):

  • Is there something like the dask-cudf integration (Multi-GPU with Dask-cuDF — cudf 0.15.0 documentation) for ray and cudf and if that makes sense?
  • Enable GPU support in Ray [1]. Just use Mars/Modin(sorry, I don’t know the difference) on top of it. That should be equivalent to dask-cudf?
  • There is a Dask on Ray :). Can I use dask-cudf while running Dask on Ray?
  • Use cuDF within ray.remote functions

Do you know of any benchmarks for these or initial guesses on which would perform faster?

[1] - GPU Support — Ray v2.0.0.dev0

Hey @devin-petersohn, do you know what 's the current status of Modin + CuDF?

Also @Clark_Zinzow can we use the cuDF with dask on ray? My impression is there’s no reason this won’t work unless dask distributed is doing something special

It’s coming along. We have excellent early performance numbers. It’s not in the main branch of Modin yet because it’s quite early, but if anyone is interested in trying it I can point them in the right direction. It’s still open source, just needs to be installed differently.

2 Likes

@devin-petersohn We would be very interested. However, we don’t need it now. We can wait. Is there a particular future version that you see this functionality in? or in other words, where do we go to find out that it’s in Ray (e.g release notes, mailing list)

You can follow Modin release notes: Releases · modin-project/modin · GitHub

Modin releases independently from Ray.