# The heartbeat between the worker and the header has failed **URL:** \ **Category:** Ray Clusters\ **Created:** [July 15, 2024, 7:09am UTC](https://discuss.ray.io/t/the-heartbeat-between-the-worker-and-the-header-has-failed/15227 "2024-07-15T07:09:48Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Dunty\_Z](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/dunty_z/32/6309_2.png) [@Dunty\_Z](https://discuss.ray.io/u/Dunty_Z)\ **Post date:** [July 15, 2024, 7:09am UTC](https://discuss.ray.io/t/the-heartbeat-between-the-worker-and-the-header-has-failed/15227/1 "2024-07-15T07:09:48Z")
- High: It blocks me to complete my task. Hi hi, I hope you can help me with the following error: “The node with node id: \* and address: \* and node name: \* has been marked dead because the detector has missed too many heartbeats from it. This can happen when a (1) raylet crashes unexpectedly (OOM, preempted node, etc.) (2) raylet has lagging heartbeats due to slow network or busy workload.” How should I resolve this issue? It seems like the heartbeat connection between the worker and the header is failing. Which port do I need to ensure is accessible for the worker to communicate with the header? Also, how do I set this port when starting the worker and the header? Thank you for your assistance. BR ---
**Author:** ![Sam\_Chan](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/sam_chan/32/5011_2.png) [@Sam\_Chan](https://discuss.ray.io/u/Sam_Chan)\ **Post date:** [July 15, 2024, 4:28pm UTC](https://discuss.ray.io/t/the-heartbeat-between-the-worker-and-the-header-has-failed/15227/2 "2024-07-15T16:28:16Z")
Does this occur after running for a while or even right on new Ray Worker startup? ---
**Author:** ![Dunty\_Z](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/dunty_z/32/6309_2.png) [@Dunty\_Z](https://discuss.ray.io/u/Dunty_Z)\ **Post date:** [July 16, 2024, 6:12am UTC](https://discuss.ray.io/t/the-heartbeat-between-the-worker-and-the-header-has-failed/15227/3 "2024-07-16T06:12:56Z")
It happened at boot time. I think maybe there’s a port that worker can’t be accessed I would like to ask you how the system works. After the header starts the program and the worker accesses the TCP server through one of the ports, the heartbeat is established in this service, or does the worker start a server and ask the header to connect? ---
**Author:** ![Sam\_Chan](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/sam_chan/32/5011_2.png) [@Sam\_Chan](https://discuss.ray.io/u/Sam_Chan)\ **Post date:** [July 16, 2024, 4:36pm UTC](https://discuss.ray.io/t/the-heartbeat-between-the-worker-and-the-header-has-failed/15227/4 "2024-07-16T16:36:15Z")
It goes Ray Head \> Worker; it’s a Driver \> Slave distributed paradigm. So your Compute fleet would need all ports available and open in order for communications to work. What’s your Compute infra setup look like? Are you running on Amazon/Google/one-of-the-cloud-providers? ---
**Author:** ![Dunty\_Z](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/dunty_z/32/6309_2.png) [@Dunty\_Z](https://discuss.ray.io/u/Dunty_Z)\ **Post date:** [July 17, 2024, 6:09am UTC](https://discuss.ray.io/t/the-heartbeat-between-the-worker-and-the-header-has-failed/15227/5 "2024-07-17T06:09:20Z")
Yes, my header is on AWS server, but I don’t want to allow access to all ports. Could you please let me know which specific ports I need to allow? ---
**Author:** ![Sam\_Chan](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/sam_chan/32/5011_2.png) [@Sam\_Chan](https://discuss.ray.io/u/Sam_Chan)\ **Post date:** [July 17, 2024, 6:34am UTC](https://discuss.ray.io/t/the-heartbeat-between-the-worker-and-the-header-has-failed/15227/6 "2024-07-17T06:34:29Z")
[https://docs.ray.io/en/master/ray-core/configure.html#ports-configurations](https://docs.ray.io/en/master/ray-core/configure.html#ports-configurations)