FileNotFoundError: [WinError 206] The filename or extension is too long

Ray Libraries (Data, Train, Tune, Serve) Ray Tune
1

Running Ray Tune (latest 2.0.0dev) on Windows, I get the following os error:

2021-04-13 14:07:07,591 ERROR trial_runner.py:732 – Trial run_search_distributed_tune_60b39_00000: Error processing event.
Traceback (most recent call last):
File “C:\Users\dm57337.conda\envs\py38tf\lib\site-packages\ray\tune\trial_runner.py”, line 702, in _process_trial
results = self.trial_executor.fetch_result(trial)
File “C:\Users\dm57337.conda\envs\py38tf\lib\site-packages\ray\tune\ray_trial_executor.py”, line 686, in fetch_result
result = ray.get(trial_future[0], timeout=DEFAULT_GET_TIMEOUT)
File “C:\Users\dm57337.conda\envs\py38tf\lib\site-packages\ray_private\client_mode_hook.py”, line 62, in wrapper
return func(*args, **kwargs)
File “C:\Users\dm57337.conda\envs\py38tf\lib\site-packages\ray\worker.py”, line 1448, in get
raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(TuneError): ray::ImplicitFunc.train_buffered() (pid=36392, ip=10.240.194.92)
File “python\ray_raylet.pyx”, line 505, in ray._raylet.execute_task
File “python\ray_raylet.pyx”, line 449, in ray._raylet.execute_task.function_executor
File “C:\Users\dm57337.conda\envs\py38tf\lib\site-packages\ray_private\function_manager.py”, line 566, in actor_method_executor
return method(__ray_actor, *args, **kwargs)
File “C:\Users\dm57337.conda\envs\py38tf\lib\site-packages\ray\tune\trainable.py”, line 173, in train_buffered
result = self.train()
File “C:\Users\dm57337.conda\envs\py38tf\lib\site-packages\ray\tune\trainable.py”, line 232, in train
result = self.step()
File “C:\Users\dm57337.conda\envs\py38tf\lib\site-packages\ray\tune\function_runner.py”, line 366, in step
self._report_thread_runner_error(block=True)
File “C:\Users\dm57337.conda\envs\py38tf\lib\site-packages\ray\tune\function_runner.py”, line 512, in _report_thread_runner_error
raise TuneError(
ray.tune.error.TuneError: Trial raised an exception. Traceback:
ray::ImplicitFunc.train_buffered() (pid=36392, ip=10.240.194.92)
File “C:\Users\dm57337.conda\envs\py38tf\lib\site-packages\ray\tune\function_runner.py”, line 248, in run
self._entrypoint()
File “C:\Users\dm57337.conda\envs\py38tf\lib\site-packages\ray\tune\function_runner.py”, line 315, in entrypoint
return self._trainable_func(self.config, self._status_reporter,
File “C:\Users\dm57337.conda\envs\py38tf\lib\site-packages\ray\tune\function_runner.py”, line 580, in _trainable_func
output = fn()
File “C:\dev\clientactivity\find_best_model.py”, line 45, in run_search_distributed_tune
with tune.checkpoint_dir(step=start) as checkpoint_dir:
File “C:\Users\dm57337.conda\envs\py38tf\lib\contextlib.py”, line 113, in enter
return next(self.gen)
File “C:\Users\dm57337.conda\envs\py38tf\lib\site-packages\ray\tune\session.py”, line 174, in checkpoint_dir
_checkpoint_dir = _session.make_checkpoint_dir(step=step)
File “C:\Users\dm57337.conda\envs\py38tf\lib\site-packages\ray\tune\function_runner.py”, line 190, in make_checkpoint_dir
checkpoint_dir = TrainableUtil.make_checkpoint_dir(
File “C:\Users\dm57337.conda\envs\py38tf\lib\site-packages\ray\tune\utils\trainable.py”, line 121, in make_checkpoint_dir
os.makedirs(checkpoint_dir, exist_ok=True)
File “C:\Users\dm57337.conda\envs\py38tf\lib\os.py”, line 223, in makedirs
mkdir(name, mode)
FileNotFoundError: [WinError 206] The filename or extension is too long: ‘C:\Users\dm57337\AppData\Local\Temp\loanrecommender_hyperparam_search_with_ray_tune\run_search_distributed_tune_60b39_00000_0_activation=tanh,batch_size=512,dot_product=True,dropout=0.55357,include_branch_concat_in_2021-04-13_14-07-04\checkpoint_000000’

How can this be resolved?

2

cc @amogkam Can you address this question?

3

@diman82 what does your tune.run look like?

4

Same problem here, how can I change the filename?

5

@Matt1 On windows, you can increase the path length limitation

6

stumbled over the same issue. By setting many parameters in Tune results in long filenames. It is very frustrating to find out, at the end of the training, that no policy checkpoint was saved because of that…
even more frustrating to realise, that there is no extern way to manipulate this path.
In my case, it was also not possible to turn on the LongPathLimitation because of user policy restrictions.

Does anyone find any workarounds for this?

7

did you find any solution?

8

Anyone tried using Longpath tool?

9

Try using Longpath Tool.