Hello!
So I have a local module that’s structured as below :
* "my_ml_module"
* __init__.py
* scripts/
* my_scripts.py
* __init__.py
* routines/
* __init__.py
* useful_routines.py
In my_scripts.py
, it may have import statements such as from my_ml_module.routines.useful_routines import abc
.
I have a EC2 autoscaling scaler where I specified my_ml_module
as file_mounts
, so the cluster nodes do have the my_ml_module
under directory /
.
Now my question is how do I submit the my_scripts.py
to the cluster, and have the cluster refer to the script under the mounted location so that PYTHONPATH
works out?
Currently when I do ray submit my_ml_module/scripts/my_scripts.py
, I think the new copy of the script is getting synced to the os.path.join(“~”, os.path.basename(script))
as specified in API and Package Reference — Ray v2.0.0.dev0 and I am getting ModuleNotFoundError: No module named my_ml_module.routines.useful_routines
.
Thanks!