Object spilling - Error cleaning up spill files

Ray Libraries (Data, Train, Tune, Serve) Ray Data
1

Hello everyone,

I am doing distributed data loading with Ray Data. However I got this error during the repartition step, that can not delete the spill file. I’m use Ray 2.0 on Windows 10

 ERROR external_storage.py:353 -- Error cleaning up spill files. You might still have remaining spilled objects inside `ray_spilled_objects` directory.
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\hung.bui\\AppData\\Local\\Temp\\ray\\session_2022-08-25_23-51-44_319651_13064\\ray_spilled_objects\\745080e71b5a40469bd78216635ff051-multi-6'

Is there any idea to solve this error? Thank you!

2

Hey @hungbui are you still running into this issue? Is it consistently reproducible?