It’s okay. I tried that and was getting the same error of this topic: https://discuss.ray.io/t/importerror-cannot-import-name-paramspec-from-typing-extensions-when-creating-a-cluster-on-azure/6869/9
Anyway, I found a combination of azure libraries that works for me:
head_setup_commands:
- pip install -U azure-cli-core==2.22.0 azure-mgmt-compute==17.0.0b1 azure-mgmt-msi==1.0.0 azure-identity==1.6.1 azure-mgmt-network==19.0.0
With this line in the yaml everything works fine and it autoscales for other worker nodes!
Thank you for the help