Hi, I met a problem when running a ray program in docker and checking the dashboard in the host.
I run a docker with docker run -p 9999:9999 option.
I use ray.init(dashboard_port=9999,dashboard_host='0.0.0.0') in the docker and the console prints INFO services.py:1270 -- View the Ray dashboard at http://172.17.0.2:9999, which is expected.
In the docker I check the dashboard is working:
root@03dc0ca63666:/home/game# lsof -i:9999
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
python3 192 root 27u IPv4 165722973 0t0 TCP *:9999 (LISTEN)
Now I want to check the dashboard in the host. If all works right, I could open localhost:9999 in the host. However, it failed, and on host I check whether the port forwarding succeed. lsof -i:9999 gives empty output.
Besides, I use docker ps to confirm that:
$docker ps
fd54703aabb2 test "bash" About a minute ago Up About a minute 0.0.0.0:8265->8265/tcp, :::8265->8265/tcp
oddly enough I’m seeing something similar, even on the default port 8265 and if I try to access the ray dashboard, I get an ERR_CONNECTION_RESET in both docker and without docker
@LoveRay I had to downgrade from aioredis 2.0.0 to aioredis@1.3.1 and then when I cleared by browser cache, I was able to get the dashboard to work again. YMMV, bu hope that helps or gives you something else to try
@puntime_error Thanks for the reply. I have confirmed that worked. The only strange thing is even if I could access host_ip:9999 for the dashboard, lsof -i:9999 gives nothing. But anyway, this is the workout.
I encounter the same question. I construct a on-primise cluster with two physical servers. docker ray-head is run on server one, and docker ray-worker is run on server two. I build network connection by docker swarm. and I can get cluster information by command “ray status”. But I cannot visit Ray web UI when I enter http://:8265 on my laptop chrome
I make the port map too.
But if when I run container without “–network” configuation, I can see Ray web UI, in that case , my other server cannot join the Ray cluster.
Is there any help?
Thanks in advance
@LoveRay and @zhonglism (saw your other post), I FINALLY got the dashboard and the metrics working in my env. I had to manually install the deps listed in <python_install>/site-packages/ray/dashboard/optional_deps.py
I didn’t need to install all of them, but add to install aiohttp_cors, prometheus_client, opencensus after that I could see get to the dashboard…ymmv, but good luck