Java to Python data transfer

Ray Libraries (Data, Train, Tune, Serve) Ray Data
1

How severe does this issue affect your experience of using Ray?

  • Medium: It contributes to significant difficulty to complete my task, but I can work around it.

I am looking to use both Python and Java with Ray. Right now this is primarily because some of my data sources are highly optimized with JDBC and I can’t get near the performance in Python. There doesn’t seem to be any native way to do this right now as the Java to Python serialization is pretty limited, Cross-Language Programming — Ray 2.5.1. Hence, Java can be used with Ray, just so long as you don’t need to move any non-trivial data between the two or handle the serialization/deserialization yourself. I’m looking at using Apache Arrow and wondering if anyone has done something like this or there are any plans for supporting something like this in Ray in the future.

Thanks,
Patrick

2

Hi @Patrick_S, yeah the Ray Data library is purely in Python now. For your data sources highly optimized with JDBC, are you reading from SQL databases?

In case it’s useful, Ray Data library supports reading from a dozen of SQL database, as long as they have a Python DB API2 connector. Some examples in Loading Data — Ray 2.5.1 .