Ray now supports running ray on spark. The documentation for this can be found here :
https://docs.ray.io/en/latest/cluster/vms/user-guides/community/spark.html
Does anyone know why the support for this is limited to standalone and databricks clusters ? Is there any reason why this would not work on cloudera / yarn ?