Error running tune on k8s


I deployed ray cluster to my k8s, following instructions on your website Deploying on Kubernetes — Ray v2.0.0.dev0.

the deployment succeeded without any errors

When I try to run simple rl app that uses tune on the cluster from my local machine, I get the following error:

 return self._call_schedule_for_task(task)
  File "/home/goranj/miniconda3/envs/rayenv/lib/python3.7/site-packages/ray/util/client/", line 318, in _call_schedule_for_task
    raise cloudpickle.loads(ticket.error)
ImportError: cannot open shared object file: No such file or directory

I am getting the same error when deploying ray to my local cluster and to AKS.

App that I am trying to run:

"env": env,
"timesteps_per_iteration": 300,
"monitor": False,
"log_level": "DEBUG",
"ignore_worker_failures": True

Thanks, tracking here:

Can you check which versions of Ray (and other dependencies) you are using on the client machine and the pods?

Any such helpful details appreciated on the issue :slight_smile:


I used latest stable Ray version, 1.4 and the helm chart to deploy Ray ( ray/deploy/charts/ray at master · ray-project/ray (