Runtime Env Agent timed out as NotFound in 30000ms. Status: NotFound: on_connect Connection refused, address: x.x.x.x, port: 19124, Suiciding...
or
> docker compose exec node serve start --proxy-location EveryNode \
--http-host 0.0.0.0 --http-port 8099
2023-12-26 10:34:06,574 INFO worker.py:1540 -- Connecting to existing Ray cluster at address: HEAD_IP:6379...
2023-12-26 10:34:06,591 INFO worker.py:1715 -- Connected to Ray cluster. View the dashboard at http://HEAD_IP:8265
[2023-12-26 10:34:15,600 E 252 282] core_worker_process.cc:216: Failed to get the system config from raylet because it is dead. Worker will terminate. Status: GrpcUnavailable: RPC Error message: failed to connect to all addresses; last error: UNKNOWN: ipv4:CURRENT_EXTERNAL_IP_OF_NODE:35369: Failed to connect to remote host: Connection refused; RPC Error details: .Please see `raylet.out` for more details.