Copy entire package to runtime env

Ray Core Ray Workflows
1

How severe does this issue affect your experience of using Ray?

  • Medium: It contributes to significant difficulty to complete my task, but I can work around it.

I’m developing a package which uses Ray workflows. It has the following structure:

my_package/
    my_package/
        subpackage_a/
            module_1.py
             __init__.py
        subpackage_b/
            module_2.py
             __init__.py 
       module_3.py
        __init__.py

In another script, I’ve written some code which invokes ray.init and runs some of the functions in my_package.

Using the runtime_env kwarg of ray.init, what’s the correct way to specify that I want this entire package to be copied to each of the Ray workers (to allow for quick development)?

Would it just be runtime_env={"py_modules": ["path/to/my_package/my_package"]}? Or would I need to add one path for each module, so something like this: runtime_env={"py_modules": ["path/to/my_package/my_package/subpackage_a/module_1.py", "path/to/my_package/my_package/subpackage_b/module_2.py", ...]}?

Thanks a lot

2

Hi @Jack_Dry

Could you try both and see which one works?

3

Hi I did try both. The first way works, i.e., specifying: path/to/my_package/my_package. I originally just had path/to/my_package which wasn’t working. Thanks!