Ray Cluster K8s: rayproject/ray-ml-gpu image - tensorflow-gpu requirement (MacOS)

I have a local macOS machine, where I connect to my ray cluster through the ray client to run tune experiments. I am using tensorflow==2.4.1 locally.
I wanted to use the provided rayproject/ray:nightly-gpu image for my head and worker nodes. However, I realized that the image has the requirement tensorflow-gpu>=2.4.0, which is not supported on macOS. Therefore, the cluster refuses to run my script.
Since tensorflow on macOS does not provide GPU support, does this mean I will not be able to submit tensorflow tuning runs on my cluster using GPUs? Would there be a way around this?

Hey Abaschkim,

as long as the computation graph is constructed inside of a task or actor, it should be possible to drive a GPU computation from a CPU client. Let us know if this works in your case and what your experience with this setup is!

Best,
Philipp.

1 Like