Publish dashboard port (aka. how to provide docker options to head node runned in container)

Ray Clusters
1

Hi all!

I’m trying to understand cluster yaml config (and in general the way the head node docker setup). This is my case: I’ve started the head with some custom dashboard IP:PORT setup:

obraz
obraz1447×133 63.3 KB

The config in docker section contains -p PORT:PORT section as follows:

obraz
obraz741×252 77.3 KB

However the ports are not published. I see some messages saying, that Published ports are discarded when using host network mode. But… I can not see any entry enabling host mode…???

Also with --dashboard-host=0.0.0.0 --dashboard-port=8266 dashboard binds to docker container IP:

obraz
obraz924×45 42.1 KB

When I try to bind dashboard to my host IP --dashboard-host=172.16.1.2 --dashboard-port=8266` dashboard does not start:

obraz
obraz1119×174 74.7 KB

I would appreciate any hints. Thanks!

How severe does this issue affect your experience of using Ray?

  • High: It blocks me to complete my task.
2

Hi @ijrsvt.
Could you please have a look?

3

@sebzur This sounds annoying.
Would you mind opening a bug report with a detailed reproduction of the issue?

cc @ckw017
Also, @ijrsvt do you have ideas on this one?

4

--net=host is automatically passed ray/docker.py at 7f1bacc7dc9caf6d0ec042e39499bbf1d9a7d065 · ray-project/ray · GitHub
@sebzur What happens if you don’t pass -p 8266:8266 in the run_options?

5

Sure, tomorrow I’ll open the related issue and I’ll describe the details.

6

Thx! Now I see that host entry!

And as for your question - without port publishing entry (-p 8266:8266 not present in run_options) dashboard is still having issues while binding to local host IP.

obraz
obraz1269×159 52.5 KB

But… Now I see that it’s (of course) OK - when running in container I can not bind to my local host IP, I should rather use container IP through $RAY_HEAD_IP. So having the following:

obraz
obraz1441×136 49.3 KB

I get the dashboard connected do container (ray head) IP and specified port, but… It’s not in my host network… (it’s having docker default subnet) - while in --net=host it should be (as far as a understand things correctly).

So I can not access it from my host (without port being published - which is not possible in host mode).