So I have a local module that’s structured as below :
* "my_ml_module" * __init__.py * scripts/ * my_scripts.py * __init__.py * routines/ * __init__.py * useful_routines.py
my_scripts.py, it may have import statements such as
from my_ml_module.routines.useful_routines import abc.
I have a EC2 autoscaling scaler where I specified
file_mounts, so the cluster nodes do have the
my_ml_module under directory
Now my question is how do I submit the
my_scripts.py to the cluster, and have the cluster refer to the script under the mounted location so that
PYTHONPATH works out?
Currently when I do
ray submit my_ml_module/scripts/my_scripts.py, I think the new copy of the script is getting synced to the
os.path.join(“~”, os.path.basename(script)) as specified in API and Package Reference — Ray v2.0.0.dev0 and I am getting
ModuleNotFoundError: No module named my_ml_module.routines.useful_routines.