Connecting Ray client to k8 cluster using ingress

I am using Ray v2.0.0 running it on the cluster (tried both kind and IBM IKS). I install ray using operator and then create an ingress for the port 10001. I can port-forward the service and it works, but using ingress will not connect. I tried both ray.init("ray://") and

This used to work a couple of month ago, but not any more

And after digging around here is the answer. Ray client is using insecure gRPC, while Nginx ingress controller only supports secure gRPC by default. It requires special configuration to support insecure gRPC Load balancing gRPC service with Nginx - DEV Community