Using conda in runtime_env

Ray Core
1

How severe does this issue affect your experience of using Ray?

  • High: It blocks me to complete my task.

(I am using ray 2.2.0)
Hi. I am quite new to ray and am setting up a first ray entrypoint. I have created a docker image with a conda environement installed in it that I want to activate when I submit my jobs through the JobSubmissionClient. When I do this my job gets stuck on PENDING and never starts. I think this is because of mismatching python/ray version between the cluster and my local machine, because the documentation states:
" conda environments must have the same Python version as the Ray cluster. Do not list ray in the conda dependencies, as it will be automatically installed."
(Environment Dependencies — Ray 2.2.0)

I am launching the job from the same environment I want to run on the cluster but this do not work. When the documentation here states the “cluster” do they refer to what is installed outside of conda, in the conda base environement or the environment I want to run?

2

Hi @AxelN, that comment only applies if you’re using runtime_env with the "conda" field. Are you using runtime environments? If so, can you see if there are any details in dashboard_agent.log or runtime_env_setup_*.log about the runtime env setup? By default these logs are located at /tmp/ray/session_latest/logs.

3

If you don’t need to specify different environments at runtime, you don’t need to use runtime_env. If the Ray cluster was started (i.e. ray start was called) inside the desired conda env, then any jobs submitted will automatically use that conda env. So perhaps you could set up your docker image entrypoint to activate the conda env before calling ray start.

4

Thanks. Yes I am using a runtime_env because I have to use conda (some packages I am using are not available in pip) . But using ray start instead could then be an option so will try that, thanks for the help!