Ray is creating hundreds of logs files under /tmp/ray/session_latest/logs/ causing disk space issue and I/O Spikes

How severe does this issue affect your experience of using Ray?

  • High: It blocks me to complete my task.

Ray v2.6.3, Ray is creating hundreds of files under /tmp/ray/session_latest/logs/ worker-*.err, worker-*.out and python-core-worker-*.logs.
I would like to understand how do I control the Event stats log printed by ray. Also I could see lot of noise in the logs due to RAY_LOG(INFO).

How do I completely disable these logs ? and also stop ray from creating so many log files, It’s causing a lot of IO spikes which I want to avoid. Could someone please help me asap ?

Are you still seeing this issue in Ray 2.7.1? We’re adding logging improvements to future versions of Ray, so those will also help with your issue.