Dashboard and monitor logging level issue

Hi Ray Team,

I set export RAY_BACKEND_LOG_LEVEL=error since logging-level option is totally ignored by ray start. However, I noticed that every 5secs the following INFO level message gets filled in the dashboard.log

2022-01-10 21:54:41,673	INFO node_head.py:270 -- Received a log for 10.244.2.36 and autoscaler
2022-01-10 21:54:46,720	INFO node_head.py:270 -- Received a log for 10.244.2.36 and autoscaler
2022-01-10 21:54:51,767	INFO node_head.py:270 -- Received a log for 10.244.2.36 and autoscaler
2022-01-10 21:54:56,713	INFO node_head.py:270 -- Received a log for 10.244.2.36 and autoscaler

The same issue with monitor.log file. Both log files are growing and do not respect log_level. These log message are not helpful. Please, let me know how to disable it. Thanks.

@sangcho: Can you review this issue and let me know I should file a bug in GitHub? Please, note that using latest ray 1.9.1 release however the same issue with previous versions.

This is a known issue. We propagate the env var option only to the cpp components, not Python (log monitor, monitor and dashboard are python components). Please feel free to contribute! It should be pretty straightforward.