How to trace back why (Rollout)Workers died?

Monitoring & Debugging
1

How severe does this issue affect your experience of using Ray?

  • Low: It annoys or frustrates me for a moment.

So far, during training with 8 (Rollout)Workers two of them died. One already after a few thousand steps, the other after more than 1m steps.
Is there a way to trace back why these (Rollout)Workers died, e.g. in any logs in the Ray Dashboard?

2

@klausk55 when some worker has error or down, there is information link with error log in console.
You can copy path to this file and read it.

3

But what if I don’t have the console output with the information link (stdout stream is my terminal/console)?
Can I find the error log somewhere in the Ray Dashboard Logs?