# Using custom path to conda installation with \`runtime\_env\`

**URL:** <https://discuss.ray.io/t/using-custom-path-to-conda-installation-with-runtime-env/3566>\
**Category:** Ray Clusters\
**Created:** [September 20, 2021, 12:19am UTC](https://discuss.ray.io/t/using-custom-path-to-conda-installation-with-runtime-env/3566 "2021-09-20T00:19:57Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![seyuboglu](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/seyuboglu/32/1290_2.png) [@seyuboglu](https://discuss.ray.io/u/seyuboglu)\
**Post date:** [September 20, 2021, 12:19am UTC](https://discuss.ray.io/t/using-custom-path-to-conda-installation-with-runtime-env/3566/1 "2021-09-20T00:19:57Z")

</div>

I’m setting up a ray Kubernetes cluster. I have conda environments installed on a persistent disk that I’d like my workers to use. I’m trying to use the `runtime_env` feature to activate these environments with:

```python
ray.init(
        "ray://<cluster-ip>:10001",
        runtime_env={
            "conda": "domino"
        },
    )

```

However, I get `RuntimeError: Starting up Server Failed! Check `ray\_client\_server\_[port].err` on the cluster.`  
When I check the logs, I see ` Could not find conda environment: domino`.

This is unsurprising – `CONDA_EXE="/home/ray/anaconda3/bin/conda"` on the workers, while my environments live in a conda installation at `/pd/common/envs/conda/bin/conda` (note: I’ve confirmed that `pd` is properly mounted).

My question is: how can I specify a custom path to a conda installation when using `runtime_env`?

I’ve tried, to no avail, setting the environment variables `RAY_CONDA_HOME` and `CONDA_EXE` in the `runtime_env` dictionary:

```auto
runtime_env={
            "env_vars": {
                "RAY_CONDA_HOME": "/home/common/envs/conda/bin/conda",
                "CONDA_EXE": "/home/common/envs/conda/bin/conda"
            },
            "conda": "domino"
        },

```

Any help would be appreciated!

Thanks!

---

<div class="post-metadata">

**Author:** ![seyuboglu](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/seyuboglu/32/1290_2.png) [@seyuboglu](https://discuss.ray.io/u/seyuboglu)\
**Post date:** [September 20, 2021, 4:41pm UTC](https://discuss.ray.io/t/using-custom-path-to-conda-installation-with-runtime-env/3566/2 "2021-09-20T16:41:11Z")

</div>

I just tried creating a custom image that sets these environment variables.  
Dockerfile:

```auto
FROM rayproject/ray:nightly-py38

ENV CONDA_EXE=/pd/common/envs/conda/bin/conda
ENV RAY_CONDA_HOME=/pd/common/envs/conda

RUN echo 'export PATH=/pd/common/envs/conda/bin:$PATH' >> /home/ray/.bashrc
RUN echo 'export CONDA_EXE=/pd/common/envs/conda/bin/conda' >> /home/ray/.bashrc

```

Using this image for the head and the worker solves the above issue: the right conda installation is being used and the `domino` environment presumably is found.

However, in the actual remote function, the wrong environment seems to be activated.

```auto
@ray.remote(
    runtime_env={"conda": "domino"},
)
def gethostname(x):
    import os
    import platform
    import sys
    import time
    print(sys.executable)
    time.sleep(0.01)
    return x + (platform.node(),)

```

This prints the base conda python, not the one for the “domino” environment.

In addition, the new image I built is causing other weird behavior. For example, I’m unable to connect to the ray dashboard:

```auto
E0920 18:10:37.437134 47273 portforward.go:400] an error occurred forwarding 8265 -> 8265: error forwarding port 8265 to pod 75d79cf91892e52a742a66027df53604349b75f3390413bc4dd2f5aa9e735638, uid : failed to execute portforward in network namespace "/var/run/netns/cni-b1d3fa65-f13c-e388-705e-c1ffb7a30c52": failed to dial 8265: dial tcp4 127.0.0.1:8265: connect: connection refused
E0920 18:10:37.438783 47273 portforward.go:400] an error occurred forwarding 8265 -> 8265: error forwarding port 8265 to pod 75d79cf91892e52a742a66027df53604349b75f3390413bc4dd2f5aa9e735638, uid : failed to execute portforward in network namespace "/var/run/netns/cni-b1d3fa65-f13c-e388-705e-c1ffb7a30c52": failed to dial 8265: dial tcp4 127.0.0.1:8265: connect: connection refused

```

**What is the best way to get custom code dependencies on the workers?**

In the ray documentation it explains “To achieve this, you can build a custom container image, using one of the [official Ray images](https://hub.docker.com/r/rayproject/ray) as the base.” But I’m unsure how to do build such an image, given that the simple image I created above is causing unexpected issues that are hard to debug.