Error when installing dependencies for worker nodes

Hi, I’m facing a problem when installing a company internal pypi package on worker nodes when initializing Ray. That’s being said, there is a --index-url in my requirement.txt. Can someone take a look that am I using the pip argument correctly?

I tried to use

         runtime_env={"working_dir": "/path/to/files/", "excludes": [".git"], "pip":"/path/to/files/requirements.txt"})

init Ray with dependencies, I got

~/.pyenv/versions/3.7.7/envs/py377/lib/python3.7/site-packages/ray/_private/runtime_env/ in _rewrite_pip_list_ray_libraries(pip_list)
    113     result = []
    114     for specifier in pip_list:
--> 115         requirement = Requirement.parse(specifier)
    116         package_name =
    117         if package_name == "ray":

~/.pyenv/versions/3.7.7/envs/py377/lib/python3.7/site-packages/pkg_resources/ in parse(s)
   3145     @staticmethod
   3146     def parse(s):
-> 3147         req, = parse_requirements(s)
   3148         return req

~/.pyenv/versions/3.7.7/envs/py377/lib/python3.7/site-packages/pkg_resources/ in parse_requirements(strs)
   3092             except StopIteration:
   3093                 return
-> 3094         yield Requirement(line)

~/.pyenv/versions/3.7.7/envs/py377/lib/python3.7/site-packages/pkg_resources/ in __init__(self, requirement_string)
   3101             super(Requirement, self).__init__(requirement_string)
   3102         except packaging.requirements.InvalidRequirement as e:
-> 3103             raise RequirementParseError(str(e))
   3104         self.unsafe_name =
   3105         project_name = safe_name(

RequirementParseError: Invalid requirement, parse error at "'--index-'"

My requirments.txt looks like

--index-url https://some/internal/pypi/server

I found this on ray’s source code, but not quite understand how does Ray handle --index-url in requirements.txt

Hi @Tian_Sang, sorry you ran into this! This is a bug in Ray 1.10. Can you try with Ray 1.9 or Ray 1.11.0rc0?

Any requirements.txt file will work, and if you instead choose to pass in a list, the lines will internally be concatenated into a requirements.txt file and pip install -r requirements.txt will be called.

Sure, I’m gonna try that

Thanks @architkulkarni! I’m working with @Tian_Sang on this issue.

Does this mean we will need to update the Ray version both on the client side (driver) and the server side (Ray cluster)? Will version mismatch between client and server be an issue? I’m also curious if there’s a workaround for the existing 1.10 version?

A workaround is to set PIP_EXTRA_INDEX_URL on the cluster before starting Ray, see

If you upgrade, you need the same Ray version on the client and server.