Is it possible to use Ray in a Singularity container on a SLURM cluster (I cannot install Ray on the cluster itself)? If so, is there any reference on how to do that?
Thanks in advance.
Could you install ray in an anaconda environment in the cluster?
Both ray and anaconda run entirely in user space and can be installed in a user’s home or work directory.
This is what I do on the HPC systems I work with.
Thank you, I will give this a try.
In the meantime, I made it work by simply calling “singularity exec … python3 script_with_ray.py” in the slurm submission script; I just had to bind some folders. It was easier than the proposed way (Deploying on Slurm — Ray v1.6.0), but since it’s not broken I won’t fix it 