Will ray worker install python packages in the run time?

ray job submit --address=‘http://address’ --runtime-env-json=‘{“working_dir”:“./” , “pip”:[“smmap”]}’ – python ./job.py

The ray worker does not have the package “smmap”. I have had used this instruction to submit a job to this ray worker, and after the submitting , I have got a error "No module named ‘smmap’ ". Does this mean ray worker will not install python packages in the run time?

If I want ray worker to install package in the run time, How can I solve it?

The python packages should get installed at run time, so it’s possible this is a bug or maybe a mistake in the API usage.

CC’ing @shrekris who can help you further.

Yes, you should have access to the smmap module at runtime. Do you mind posting a reproduction script? What are the contents of job.py?

Thanks!

This is the codes included in the job.py .

import ray
import requests
import smmap
import sys

ray.init()

@ray.remote
class Counter:
    def __init__(self):
        self.counter = 0

    def inc(self):
        print("ok")

counter = Counter.remote()

for _ in range(5):
    ray.get(counter.inc.remote())

print(sys.version)

I use this instruction to submit the job.

ray job submit --address=‘http://192.168.0.166:8265’ --runtime-env-json=‘{“working_dir”:“./” , “pip”:[“smmap”]}’ -- /home/wanjia/conda/env1/bin/python ./job.py

I submit the job from my ray node to a remote ray node(192.198.0.166:8265). And this is the error message:

Job submission server address: http://192.168.0.166:8265
2022-07-06 14:55:02,101	INFO dashboard_sdk.py:272 -- Uploading package gcs://_ray_pkg_90994eb4a30dc674.zip.
2022-07-06 14:55:02,101	INFO packaging.py:479 -- Creating a file package for local directory './'.

-------------------------------------------------------
Job 'raysubmit_z9DDa2AMUUGxzb8Z' submitted successfully
-------------------------------------------------------

Next steps
  Query the logs of the job:
    ray job logs raysubmit_z9DDa2AMUUGxzb8Z
  Query the status of the job:
    ray job status raysubmit_z9DDa2AMUUGxzb8Z
  Request the job to be stopped:
    ray job stop raysubmit_z9DDa2AMUUGxzb8Z

Tailing logs until the job exits (disable with --no-wait):
Traceback (most recent call last):
  File "./job.py", line 3, in <module>
    import smmap
ModuleNotFoundError: No module named 'smmap'

---------------------------------------
Job 'raysubmit_z9DDa2AMUUGxzb8Z' failed
---------------------------------------

Status message: Job failed due to an application error, last available logs:
Traceback (most recent call last):
  File "./job.py", line 3, in <module>
    import smmap
ModuleNotFoundError: No module named 'smmap'

Thanks for posting the repro! This does seem like a bug, since I believe smmap should be importable in this script. Do you mind filing an issue here, so we can track it?

Thanks for you reply. Yes, I do think so too! We intend to do some test to check whether ray worker will install python packages at runtime.

For this intent, we not just tested the package “smmap” which is a small package and takes only a short time to download. Correspondingly, we also took packages like numpy and torch for testing, but we got the same message.

I have posted an issue to our ray project in github.