New to the community. Wondering, is there anyway to configure Ray cluster to access to azure blob storage using managed identities?
Really appreciate your help
Ray nodes deployed to azure using the autoscaler (e.g. ray up azure_config.yml) will have a user assigned managed identity created automatically: ray-msi-user-identity if you grant that identity access to your storage account then the cluster nodes should be able to access it. How you implement pulling / pushing data can vary a bit. For example using python sdk or just pointing to a url, or some other method. if you have more specifics or example code you want some help with, I’m happy to take a look.
@gramhagen Thanks for your prompt response! However, we are currently deployed in k8s mode via the Helm chart at deploy/charts/ray. I’m wondering whether the recommendation above still work with k8s operator?
if you’re deploying to AKS it might be possible, though I have no idea how to configure AKS managed identities via the ray helm chart templates. if you’re deploying outside of azure the only way I can think of is adding a command to store a sas token as an environment variable during cluster setup.