I am doing distributed data loading with Ray Data. However I got this error during the repartition step, that can not delete the spill file. I’m use Ray 2.0 on Windows 10
ERROR external_storage.py:353 -- Error cleaning up spill files. You might still have remaining spilled objects inside `ray_spilled_objects` directory. PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\hung.bui\\AppData\\Local\\Temp\\ray\\session_2022-08-25_23-51-44_319651_13064\\ray_spilled_objects\\745080e71b5a40469bd78216635ff051-multi-6'
Is there any idea to solve this error? Thank you!