# Private python dependencies and RuntimeEnv

**URL:** <https://discuss.ray.io/t/private-python-dependencies-and-runtimeenv/10720>\
**Category:** Ray Core\
**Created:** [May 18, 2023, 5:24pm UTC](https://discuss.ray.io/t/private-python-dependencies-and-runtimeenv/10720 "2023-05-18T17:24:17Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![wizrds](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/wizrds/32/4456_2.png) [@wizrds](https://discuss.ray.io/u/wizrds)\
**Post date:** [May 18, 2023, 5:24pm UTC](https://discuss.ray.io/t/private-python-dependencies-and-runtimeenv/10720/1 "2023-05-18T17:24:17Z")

</div>

**How severe does this issue affect your experience of using Ray?**

- High: It blocks me to complete my task.

I have some python packages I’m passing to the RuntimeEnv py\_modules as wheel files, however some of these packages depend on packages from private registries. I can not figure out the best way to go about giving access to the Worker process via credentials. I tried setting the env vars `PIP_EXTRA_INDEX_URL` and `PIP_INDEX_URL` but it doesn’t seem to pass the value to the subprocess call when installing the wheel. I thought about using a requirements.txt with the information but I don’t want to hardcode sensitive information in my distribution, and can’t enforce others to do so as well. Any ideas?

---

<div class="post-metadata">

**Author:** ![wizrds](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/wizrds/32/4456_2.png) [@wizrds](https://discuss.ray.io/u/wizrds)\
**Post date:** [May 18, 2023, 6:04pm UTC](https://discuss.ray.io/t/private-python-dependencies-and-runtimeenv/10720/2 "2023-05-18T18:04:03Z")

</div>

I see a similar question [here](https://discuss.ray.io/t/adding-index-option-to-pip-runtime-env/10606/4) however from what I can tell they are using the `pip` option and not `py_modules` and from the quick skim of the code it looks like the two use different functions when calling pip install command.

---

<div class="post-metadata">

**Author:** ![cade](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/cade/32/3837_2.png) [@cade](https://discuss.ray.io/u/cade)\
**Post date:** [May 18, 2023, 8:47pm UTC](https://discuss.ray.io/t/private-python-dependencies-and-runtimeenv/10720/3 "2023-05-18T20:47:00Z")

</div>

Hi @wizrds . Can you share what happened when you tried [Adding index option to pip runtime\_env](https://discuss.ray.io/t/adding-index-option-to-pip-runtime-env/10606) ? E.g. adding `PIP_INDEX_URL` to runtime env vars? Maybe there is some workaround that unblocks you.

For secrets, there’s an open feature request for this. Would be great if you could +1/add a small blurb here [[Core]Read working\_dir zip from private s3 · Issue #34708 · ray-project/ray · GitHub](https://github.com/ray-project/ray/issues/34708)

---

<div class="post-metadata">

**Author:** ![wizrds](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/wizrds/32/4456_2.png) [@wizrds](https://discuss.ray.io/u/wizrds)\
**Post date:** [May 19, 2023, 4:25am UTC](https://discuss.ray.io/t/private-python-dependencies-and-runtimeenv/10720/4 "2023-05-19T04:25:45Z")

</div>

Ok I tried a few different things. I’m using `package-a` which has a dependency to `package-b` but `package-b` is in a private index. When I pass the `package-a` wheel file to `py_modules`, and set the pip env vars it fails with `RuntimeError: Failed to install py_modules wheel`. When I pass the `package-a` wheel, and set the `pip` param to install `package-b`, and set the env vars it still fails. It looks like it will successfully install `package-b` however it fails when trying to install the wheel. I imagine this would be because the pip install is after the py\_modules installs? Finally, when I do not pass a wheel at all, and use the `pip` param to install `package-a` directly with the pip env vars it correctly works. Unfortunately I can only rely on passing source distributions and wheels via py\_modules in my application so setting pip installs isn’t something we can do.

---

<div class="post-metadata">

**Author:** ![jjyao](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/jjyao/32/1799_2.png) [@jjyao](https://discuss.ray.io/u/jjyao)\
**Post date:** [May 24, 2023, 4:18pm UTC](https://discuss.ray.io/t/private-python-dependencies-and-runtimeenv/10720/5 "2023-05-24T16:18:54Z")

</div>

@wizrds

Do you mind filing a github enhancement ticket?

---

<div class="post-metadata">

**Author:** ![Jules\_Damji](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/jules_damji/32/4058_2.png) [@Jules\_Damji](https://discuss.ray.io/u/Jules_Damji)\
**Post date:** [May 24, 2023, 9:47pm UTC](https://discuss.ray.io/t/private-python-dependencies-and-runtimeenv/10720/6 "2023-05-24T21:47:33Z")

</div>

@wizrds Once you have filed an enhancement ticket as suggested by @jjyao, we can close this issue.

---

<div class="post-metadata">

**Author:** ![wizrds](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/wizrds/32/4456_2.png) [@wizrds](https://discuss.ray.io/u/wizrds)\
**Post date:** [May 26, 2023, 12:11am UTC](https://discuss.ray.io/t/private-python-dependencies-and-runtimeenv/10720/7 "2023-05-26T00:11:23Z")

</div>

I have already filed one [[Core] Private dependencies and `py_modules` · Issue #35559 · ray-project/ray · GitHub](https://github.com/ray-project/ray/issues/35559)
