Unclear how to deploy ray service with working_dir in GS bucket

Ray Libraries (Data, Train, Tune, Serve)
1

Hello! I’m trying to launch ray serve task in the kubernetes cluster.
I packaged working dir as a zip and uploaded it to the Google Cloud Storage bucket.
I used this as a starter for my deployment kuberay/ray_v1alpha1_rayservice.yaml at release-0.3 · ray-project/kuberay · GitHub and replaced working_dir
url with the path in google bucket.
I get the error message You must pip install smart_open and pip install google-cloud-storage to fetch URIs in Google Cloud Storage bucket. But I don’t understand where this should be executed.
I tried to add pip section into serveConfig, but it seems to be ignored

  serveConfig:
    importPath: fruit.deployment_graph
    runtimeEnv: |
      working_dir: "gs://mybucket/fruit-example.zip"
      pip:
      - google-cloud-storage==1.43.0
      - smart-open

Could someone please point me out what do I miss?

2

Hi @uthark sorry for the poor experience here. I think the problem is that if specified in the runtime_env, there is no guarantee that the pip packages will be installed before the working_dir downloading happens, since those packages are installed at runtime.

Is it possible to preinstall the pip packages on all the cluster nodes, or somehow specify them in the container image you’re using? cc @Sihan_Wang in case you already have some direct experience with this

3

Do I understand it correctly that you propose to build a new docker image on top of rayproject/ray:2.2.0-py39-gpu and add required pip packages?

4

Sorry for the delayed response, yes that would be my proposal. Basically, anything that allows smart_open and google-cloud-storage to be already installed before Ray is started.

5

Hi, how can we pip install smart_open and google-cloud-storage on the ray cluster before ray starts. (is there any option other than building a new docker image ontop of rayproject/ray:2.2.0-py39-gpu)

6

@saifjilani You will to build a docker image atop rayproject/ray:2.2.0-py39-gpu as suggested above by @architkulkarni to @uthark response and make those part of the required pip packages.

1 Like