Ray behavior with deleted log files

Ray Core
1

How severe does this issue affect your experience of using Ray?

  • Medium: It contributes to significant difficulty to complete my task, but I can work around it.

I have a long-running instance of Ray on a node with many workers (number of CPUs). Each worker has its own python-core-worker.log file, which have grown quite large in size and are taking ~17GB of disk space.

Based on the documentation, worker logs are not rotated. In order to free up space, I would like to delete some of these files. Would this cause any issues?

Would the log files be recreated as Ray continues to run? What are best-practices for managing these system files?

2

I think worker logs (prefix=worker) are not rotated, but core-worker logs (prefix=python-core-worker) are:

Have you configured log rotation properly? Configuring Logging — Ray 2.24.0