Ray serving with working directory as folder location

Ray Libraries (Data, Train, Tune, Serve) Ray Serve
1

Hi,
We are using ray serving on kubernetics. We have serving yaml for deployments with userconfig.
We have created a custom image (with source code- .py file) on top of “ray:2.0.0” and it is used against “headGroupSpec” and “workerGroupSpecs”.
But we are confused regarding “working_dir” within “runtimeEnv”.
We want to use the python file in the image, but we are not able to provide location within the image as working_dir.
It is mentioned as "working_dir: “https://github.com/ray-project/test_dag/archive/c620251044717ace0a4c19d766d43c5099af8a77.zip” in the document.
Is it possible to mention the location within the image as the working_dir(i.e folder location within image) as “working_dir:/home/ray/serving/”?
can you please give one example for “file://” as “http://” in “working_dir”?

Thanks,
Dhiman

2

A copy of the working_dir will be downloaded to the cluster at runtime, and the current working directory of each remote Ray worker will be changed to that working_dir copy. If you want to access a certain python file, you can use the file path relative to working_dir. For example, if your working_dir directly contains graph.py which contains a bound deployment node named app, you can use graph:app as your import path.

3

Hi @Ritapa_Kundu , unfortunately it is not currently possible to specify a folder location within the image in working_dir. We have an issue to track this feature request [core][runtime env] Remove code_search_path for job config and unify this functionality in Runtime Env · Issue #26784 · ray-project/ray · GitHub but we don’t have a timeline currently. Currently only Remote URIs (like the Github ZIP) are supported for Serve deployment graphs.

Is it possible for you to edit the Ray startup commands in your custom image? For example, if you cd /home/ray/serving before ray start is called, then that will be the directory all Ray process will run in.

4

Oops, just saw Cindy’s message come in while writing my message… Cindy is right, but currently for Serve deployments I believe the working_dir needs to be a Remote URI.