May I can deploy ray server by java application?

Ray Libraries (Data, Train, Tune, Serve) Ray Serve
1

I have a ray server for python model，but I want to start and deploy ray server in java application。What should I do？

2

The Java API is current work in progress, you can follow it at [WIP]Ray Serve Java API by liuyang-my · Pull Request #22726 · ray-project/ray · GitHub

4

Will this work supports the use case of java application sending python tasks to ray grid?
In the past, it seems not working properly and I got a response from the community saying the Java client api is not there yet.

5

I have to query java jdk for ray server deploy（java/serve/src/main/java/io/ray/serve/api/ClientTest.java）.
Suppose I have a Ray Server Deployment named Serving done in Python，I have a few questions.
1、After init ray cluster,Where should I specify the python path and runtimeenv to the serving depployment.
2、Config ray server host and port in python?
3、When will the changes to Java sdk for ray server be released?

Thank you for your patience

6

@blshao84 What’s in the work in that PR still assumes Java client running within the Ray Cluster. If you want the Java application to live outside the cluster, why not just send regular HTTP requests to the Serve endpoints?

@839576266 We we are enabling here is ability to use both Java and Python to configure a Ray Serve cluster. Your deployment can be in both language and they can call each other without limitation. It does require both Java and Python runtime to be present and you can use Python to bootstrap the initial runtime environment.

You can follow the work in that PR. But the timeline for the full stable release is around a full release by end of Q2.

7

@simon-mo sending requests to ray serve is exactly what I’m doing right now, it’s like java->ray serve->grid->ray serve->java. However, ray serve doesn’t really do anything in my use case, that’s why I asked whether it makes sense to skip serve endpoint.

8

I am thinking of deploying a Ray Server through Java. I want to use the Java client to manage ray Server of Python side (including starting and stopping all ray server,this is somewhat similar to spark’s kill job ), similar to process management.Does ray server java client plan to do these tasks?

9

When you say ‘Ray Server’ you actually mean ‘Ray Cluster’, right? What you want is to manage ray cluster in your Java Application?

10

When I say ray server, I mean a Server_Controller_Actor job.I want to manage all server deployments.