Support setting up PEX in runtime_env

Ray Core
1

This is a feature request for supporting PEX in the runtime_env of ray, which is similar to how conda is supported (see issue #14524).

For example, we can package an actor “foo” in a PEX file “my_package.pex”:

@ray.remote class Foo(object): ...

And then initialize ray with the PEX environment to enable the worker to run “foo” in the PEX environment:
ray.init(runtime_env = { "pex": "path/to/my_package.pex" });

We have an example repo GitHub - leozc/fp20 to package ray actors into PEX files and run the actor in a local ray cluster. The commands are:

build ray_demo.pex in dist:
./pants package ray_demo:ray_demo

run ray workers in PEX environment:
./dist/ray_demo/ray_demo.pex

What we would like to have is to execute the code in a real cluster (e.g. cloud, k8s, etc.).

2

Hi Mingshi, I just created a feature request to track this on the Ray repo:

Let’s discuss more there, but if you’d be willing to contribute this we’d be happy to help you figure out the best approach :slight_smile:

3

Thanks Edward! We’d be happy to contribute if you could help us with the right approach!