ray job submit --address=‘http://address’ --runtime-env-json=‘{“working_dir”:“./” , “pip”:[“smmap”]}’ – python ./job.py
The ray worker does not have the package “smmap”. I have had used this instruction to submit a job to this ray worker, and after the submitting , I have got a error "No module named ‘smmap’ ". Does this mean ray worker will not install python packages in the run time?
If I want ray worker to install package in the run time, How can I solve it?
import ray
import requests
import smmap
import sys
ray.init()
@ray.remote
class Counter:
def __init__(self):
self.counter = 0
def inc(self):
print("ok")
counter = Counter.remote()
for _ in range(5):
ray.get(counter.inc.remote())
print(sys.version)
I use this instruction to submit the job.
ray job submit --address=‘http://192.168.0.166:8265’ --runtime-env-json=‘{“working_dir”:“./” , “pip”:[“smmap”]}’ -- /home/wanjia/conda/env1/bin/python ./job.py
I submit the job from my ray node to a remote ray node(192.198.0.166:8265). And this is the error message:
Job submission server address: http://192.168.0.166:8265
2022-07-06 14:55:02,101 INFO dashboard_sdk.py:272 -- Uploading package gcs://_ray_pkg_90994eb4a30dc674.zip.
2022-07-06 14:55:02,101 INFO packaging.py:479 -- Creating a file package for local directory './'.
-------------------------------------------------------
Job 'raysubmit_z9DDa2AMUUGxzb8Z' submitted successfully
-------------------------------------------------------
Next steps
Query the logs of the job:
ray job logs raysubmit_z9DDa2AMUUGxzb8Z
Query the status of the job:
ray job status raysubmit_z9DDa2AMUUGxzb8Z
Request the job to be stopped:
ray job stop raysubmit_z9DDa2AMUUGxzb8Z
Tailing logs until the job exits (disable with --no-wait):
Traceback (most recent call last):
File "./job.py", line 3, in <module>
import smmap
ModuleNotFoundError: No module named 'smmap'
---------------------------------------
Job 'raysubmit_z9DDa2AMUUGxzb8Z' failed
---------------------------------------
Status message: Job failed due to an application error, last available logs:
Traceback (most recent call last):
File "./job.py", line 3, in <module>
import smmap
ModuleNotFoundError: No module named 'smmap'
Thanks for posting the repro! This does seem like a bug, since I believe smmap should be importable in this script. Do you mind filing an issue here, so we can track it?
Thanks for you reply. Yes, I do think so too! We intend to do some test to check whether ray worker will install python packages at runtime.
For this intent, we not just tested the package “smmap” which is a small package and takes only a short time to download. Correspondingly, we also took packages like numpy and torch for testing, but we got the same message.
I have posted an issue to our ray project in github.