When I run my code I get the following error: any help please?
Function checkpointing is disabled. This may result in unexpected behavior when using checkpointing features or certain schedulers. To enable, set the train function arguments to be `func(config, checkpoint_dir=None)`.
[I 2021-02-17 13:36:00,543] A new study created in memory with name: optuna
Warning: The actor ImplicitFunc has size 25927174 when pickled. It will be stored in Redis, which could cause memory issues. This may mean that its definition uses a large array or other object.
Log sync requires rsync to be installed.
== Status ==
Memory usage on this node: 13.9/15.7 GiB
Using AsyncHyperBand: num_stopped=0
Bracket: Iter 40.000: None | Iter 10.000: None
Resources requested: 6/8 CPUs, 0/0 GPUs, 0.0/1.42 GiB heap, 0.0/0.49 GiB objects
Result logdir: C:\Users\ali\ray_results\optuna_search
Number of trials: 1/2 (1 RUNNING)
(pid=14376) 2021-02-17 13:36:05.565472: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'cudart64_101.dll';
dlerror: cudart64_101.dll not found
(pid=14376) 2021-02-17 13:36:05.565769: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
(pid=14376) 2021-02-17 13:36:10.953665: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'nvcuda.dll'; dlerror: nvcuda.dll not found
(pid=14376) 2021-02-17 13:36:10.962572: W tensorflow/stream_executor/cuda/cuda_driver.cc:312] failed call to cuInit: UNKNOWN ERROR (303)
(pid=14376) 2021-02-17 13:36:10.967301: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:169] retrieving CUDA diagnostic information for host: LAPTOP-6O4H5M9S
(pid=14376) 2021-02-17 13:36:10.967503: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:176] hostname: LAPTOP-6O4H5M9S
(pid=14376) 2021-02-17 13:36:10.968858: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN)to use the following CPU instructions in performance-critical operations: AVX2
(pid=14376) To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
(pid=14376) 2021-02-17 13:36:10.993431: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x26d69dc9bd0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
(pid=14376) 2021-02-17 13:36:10.993527: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version
(pid=14376) Epoch 1/2
(pid=14376)
(pid=14376) 1/11 [=>............................] - ETA: 0s - loss: 0.7881 - accuracy: 0.4688
(pid=14376)
(pid=14376) 2/11 [====>.........................] - ETA: 2s - loss: 4.2451 - accuracy: 0.6250
(pid=14376)
(pid=14376) 3/11 [=======>......................] - ETA: 3s - loss: 4.2105 - accuracy: 0.6771
(pid=14376)
(pid=14376) 4/11 [=========>....................] - ETA: 3s - loss: 3.9206 - accuracy: 0.6953
(pid=14376)
(pid=14376) 5/11 [============>.................] - ETA: 3s - loss: 3.4530 - accuracy: 0.7063
(pid=14376)
(pid=14376) 6/11 [===============>..............] - ETA: 2s - loss: 2.9415 - accuracy: 0.7240
(pid=14376)
(pid=14376) 7/11 [==================>...........] - ETA: 2s - loss: 2.8680 - accuracy: 0.7009
(pid=14376)
(pid=14376) 8/11 [====================>.........] - ETA: 1s - loss: 2.6420 - accuracy: 0.7109
(pid=14376)
(pid=14376) 9/11 [=======================>......] - ETA: 1s - loss: 2.4364 - accuracy: 0.7257
(pid=14376)
(pid=14376) 10/11 [==========================>...] - ETA: 0s - loss: 2.2629 - accuracy: 0.7250
(pid=14376)
(pid=14376) 11/11 [==============================] - ETA: 0s - loss: 2.1905 - accuracy: 0.7217
Trial Runner checkpointing failed: [WinError 183] Cannot create a file when that file already exists: 'C:\\Users\\ali\\ray_results\\optuna_search\\.tmp_searcher_ckpt' -> 'C:\\Users\\ali\\ray_results\\optuna_search\\searcher-state-2021-02-17_13-36-00.pkl'
== Status ==
Memory usage on this node: 14.3/15.7 GiB: ***LOW MEMORY*** less than 10% of the memory on this node is available for use. This can cause unexpected crashes. Consider reducing the memory used by your application or reducing the Ray object store size by setting `object_store_memory` when calling `ray.init`.
Using AsyncHyperBand: num_stopped=0
Bracket: Iter 40.000: None | Iter 10.000: None
Resources requested: 6/8 CPUs, 0/0 GPUs, 0.0/1.42 GiB heap, 0.0/0.49 GiB objects
Result logdir: C:\Users\ali\ray_results\optuna_search
Number of trials: 1/2 (1 RUNNING)
(pid=14376)
(pid=14376) 11/11 [==============================] - 8s 714ms/step - loss: 2.1905 - accuracy: 0.7217 - val_loss: 0.5467 - val_accuracy: 0.7765
(pid=14376) Epoch 2/2
(pid=14376)
(pid=14376) 1/11 [=>............................] - ETA: 0s - loss: 0.3112 - accuracy: 0.9062
(pid=14376)
(pid=14376) 2/11 [====>.........................] - ETA: 3s - loss: 0.3835 - accuracy: 0.8438
(pid=14376)
(pid=14376) 3/11 [=======>......................] - ETA: 3s - loss: 0.5169 - accuracy: 0.8021
(pid=14376)
(pid=14376) 4/11 [=========>....................] - ETA: 3s - loss: 0.4979 - accuracy: 0.8125
(pid=14376)
(pid=14376) 5/11 [============>.................] - ETA: 3s - loss: 0.5055 - accuracy: 0.8125
(pid=14376)
(pid=14376) 6/11 [===============>..............] - ETA: 2s - loss: 0.5158 - accuracy: 0.8125
(pid=14376)
(pid=14376) 7/11 [==================>...........] - ETA: 2s - loss: 0.5819 - accuracy: 0.7902
(pid=14376)
(pid=14376) 8/11 [====================>.........] - ETA: 1s - loss: 0.5671 - accuracy: 0.7969
(pid=14376)
(pid=14376) 9/11 [=======================>......] - ETA: 1s - loss: 0.5342 - accuracy: 0.8090
(pid=14376)
(pid=14376) 10/11 [==========================>...] - ETA: 0s - loss: 0.5172 - accuracy: 0.8125
(pid=14376)
(pid=14376) 11/11 [==============================] - ETA: 0s - loss: 0.5125 - accuracy: 0.8058
== Status ==
Memory usage on this node: 14.3/15.7 GiB: ***LOW MEMORY*** less than 10% of the memory on this node is available for use. This can cause unexpected crashes. Consider reducing the memory used by your application or reducing the Ray object store size by setting `object_store_memory` when calling `ray.init`.
Using AsyncHyperBand: num_stopped=0
Bracket: Iter 40.000: None | Iter 10.000: None
Resources requested: 6/8 CPUs, 0/0 GPUs, 0.0/1.42 GiB heap, 0.0/0.49 GiB objects
Result logdir: C:\Users\ali\ray_results\optuna_search
Number of trials: 1/2 (1 RUNNING)
(pid=14376) 2021-02-17 13:36:33,587 ERROR function_runner.py:254 -- Runner Thread raised error.
(pid=14376) Traceback (most recent call last):
(pid=14376) File "C:\Users\ali\miniconda3\envs\testEnv\lib\site-packages\ray\tune\function_runner.py", line 248, in run
(pid=14376) self._entrypoint()
(pid=14376) File "C:\Users\ali\miniconda3\envs\testEnv\lib\site-packages\ray\tune\function_runner.py", line 315, in entrypoint
(pid=14376) return self._trainable_func(self.config, self._status_reporter,
(pid=14376) File "C:\Users\ali\miniconda3\envs\testEnv\lib\site-packages\ray\tune\function_runner.py", line 577, in _trainable_func
(pid=14376) handle_output(output)
(pid=14376) File "C:\Users\ali\miniconda3\envs\testEnv\lib\site-packages\ray\tune\function_runner.py", line 566, in handle_output
(pid=14376) raise ValueError(
(pid=14376) ValueError: Invalid return or yield value. Either return/yield a single number or a dictionary object in your trainable function.
(pid=14376) Exception in thread Thread-2:
(pid=14376) Traceback (most recent call last):
(pid=14376) File "C:\Users\ali\miniconda3\envs\testEnv\lib\threading.py", line 932, in _bootstrap_inner
(pid=14376) self.run()
(pid=14376) File "C:\Users\ali\miniconda3\envs\testEnv\lib\site-packages\ray\tune\function_runner.py", line 267, in run
(pid=14376) raise e
(pid=14376) File "C:\Users\ali\miniconda3\envs\testEnv\lib\site-packages\ray\tune\function_runner.py", line 248, in run
(pid=14376) self._entrypoint()
(pid=14376) File "C:\Users\ali\miniconda3\envs\testEnv\lib\site-packages\ray\tune\function_runner.py", line 315, in entrypoint
(pid=14376) return self._trainable_func(self.config, self._status_reporter,
(pid=14376) File "C:\Users\ali\miniconda3\envs\testEnv\lib\site-packages\ray\tune\function_runner.py", line 577, in _trainable_func
Trial train_fbab758b: Error processing event.
Traceback (most recent call last):
File "C:\Users\ali\miniconda3\envs\testEnv\lib\site-packages\ray\tune\trial_runner.py", line 586, in _process_trial
results = self.trial_executor.fetch_result(trial)
File "C:\Users\ali\miniconda3\envs\testEnv\lib\site-packages\ray\tune\ray_trial_executor.py", line 609, in fetch_result
result = ray.get(trial_future[0], timeout=DEFAULT_GET_TIMEOUT)
File "C:\Users\ali\miniconda3\envs\testEnv\lib\site-packages\ray\_private\client_mode_hook.py", line 47, in wrapper
return func(*args, **kwargs)
File "C:\Users\ali\miniconda3\envs\testEnv\lib\site-packages\ray\worker.py", line 1456, in get
raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(TuneError): ray::ImplicitFunc.train_buffered()(pid=14376) (pid=14376, ip=192.168.0.100)
File "python\ray\_raylet.pyx", line 480, in ray._raylet.execute_task
File "python\ray\_raylet.pyx", line 432, in ray._raylet.execute_task.function_executor
File "C:\Users\ali\miniconda3\envs\testEnv\lib\site-packages\ray\function_manager.py", line 556, in actor_method_executor
return method(__ray_actor, *args, **kwargs)
File "C:\Users\ali\miniconda3\envs\testEnv\lib\site-packages\ray\tune\trainable.py", line 167, in train_buffered
result = self.train()
File "C:\Users\ali\miniconda3\envs\testEnv\lib\site-packages\ray\tune\trainable.py", line 226, in train
result = self.step()
File "C:\Users\ali\miniconda3\envs\testEnv\lib\site-packages\ray\tune\function_runner.py", line 366, in step
self._report_thread_runner_error(block=True)
File "C:\Users\ali\miniconda3\envs\testEnv\lib\site-packages\ray\tune\function_runner.py", line 512, in _report_thread_runner_error
raise TuneError(
ray.tune.error.TuneError: Trial raised an exception. Traceback:
handle_output(output)
ray::ImplicitFunc.train_buffered() (pid=14376, ip=192.168.0.100)
File "C:\Users\ali\miniconda3\envs\testEnv\lib\site-packages\ray\tune\function_runner.py", line 248, in run
self._entrypoint()
File "C:\Users\ali\miniconda3\envs\testEnv\lib\site-packages\ray\tune\function_runner.py", line 315, in entrypoint
return self._trainable_func(self.config, self._status_reporter,
File "C:\Users\ali\miniconda3\envs\testEnv\lib\site-packages\ray\tune\function_runner.py", line 577, in _trainable_func
handle_output(output)
File "C:\Users\ali\miniconda3\envs\testEnv\lib\site-packages\ray\tune\function_runner.py", line 566, in handle_output
raise ValueError(
ValueError: Invalid return or yield value. Either return/yield a single number or a dictionary object in your trainable function.
(pid=14376) File "C:\Users\ali\miniconda3\envs\testEnv\lib\site-packages\ray\tune\function_runner.py", line 566, in handle_output
Traceback (most recent call last):
File "C:\Users\ali\miniconda3\envs\testEnv\lib\site-packages\ray\tune\trial_runner.py", line 586, in _process_trial
results = self.trial_executor.fetch_result(trial)
(pid=14376) File "C:\Users\ali\miniconda3\envs\testEnv\lib\site-packages\ray\tune\ray_trial_executor.py", line 609, in fetch_result
ValueError: Invalid return or yield value. Either return/yield a single number or a dictionary object in your trainable function.
(pid=14376)result = ray.get(trial_future[0], timeout=DEFAULT_GET_TIMEOUT)
File "C:\Users\ali\miniconda3\envs\testEnv\lib\site-packages\ray\_private\client_mode_hook.py", line 47, in wrapper
return func(*args, **kwargs)
(pid=14376) File "C:\Users\ali\miniconda3\envs\testEnv\lib\site-packages\ray\worker.py", line 1456, in get
11/11 [==============================] - 7s 627ms/step - loss: 0.5125 - accuracy: 0.8058 - val_loss: 0.4086 - val_accuracy: 0.8471
(pid=14376)raise value.as_instanceof_cause() validation_loss=0.4085962176322937
ray.exceptions.RayTaskError(TuneError): ray::ImplicitFunc.train_buffered() (pid=14376, ip=192.168.0.100)
File "python\ray\_raylet.pyx", line 480, in ray._raylet.execute_task
File "python\ray\_raylet.pyx", line 432, in ray._raylet.execute_task.function_executor
File "C:\Users\ali\miniconda3\envs\testEnv\lib\site-packages\ray\function_manager.py", line 556, in actor_method_executor
return method(__ray_actor, *args, **kwargs)
File "C:\Users\ali\miniconda3\envs\testEnv\lib\site-packages\ray\tune\trainable.py", line 167, in train_buffered
result = self.train()
File "C:\Users\ali\miniconda3\envs\testEnv\lib\site-packages\ray\tune\trainable.py", line 226, in train
result = self.step()
File "C:\Users\ali\miniconda3\envs\testEnv\lib\site-packages\ray\tune\function_runner.py", line 366, in step
self._report_thread_runner_error(block=True)
File "C:\Users\ali\miniconda3\envs\testEnv\lib\site-packages\ray\tune\function_runner.py", line 512, in _report_thread_runner_error
raise TuneError(
ray.tune.error.TuneError: Trial raised an exception. Traceback:
ray::ImplicitFunc.train_buffered() (pid=14376, ip=192.168.0.100)
File "C:\Users\ali\miniconda3\envs\testEnv\lib\site-packages\ray\tune\function_runner.py", line 248, in run
self._entrypoint()
File "C:\Users\ali\miniconda3\envs\testEnv\lib\site-packages\ray\tune\function_runner.py", line 315, in entrypoint
return self._trainable_func(self.config, self._status_reporter,
File "C:\Users\ali\miniconda3\envs\testEnv\lib\site-packages\ray\tune\function_runner.py", line 577, in _trainable_func
handle_output(output)
File "C:\Users\ali\miniconda3\envs\testEnv\lib\site-packages\ray\tune\function_runner.py", line 566, in handle_output
raise ValueError(
ValueError: Invalid return or yield value. Either return/yield a single number or a dictionary object in your trainable function.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "c:/Users/ali/OneDrive/Desktop/pythonProjects/Deep-learning-algo-master/src/myfile.py", line 279, in <module>
main()
File "c:/Users/ali/OneDrive/Desktop/pythonProjects/Deep-learning-algo-master/src/myfile.py", line 249, in main
analysis = tune.run(trainer.train,
File "C:\Users\ali\miniconda3\envs\testEnv\lib\site-packages\ray\tune\tune.py", line 421, in run
runner.step()
File "C:\Users\ali\miniconda3\envs\testEnv\lib\site-packages\ray\tune\trial_runner.py", line 402, in step
self._process_events(timeout=timeout) # blocking
File "C:\Users\ali\miniconda3\envs\testEnv\lib\site-packages\ray\tune\trial_runner.py", line 560, in _process_events
self._process_trial(trial)
File "C:\Users\ali\miniconda3\envs\testEnv\lib\site-packages\ray\tune\trial_runner.py", line 617, in _process_trial
self._process_trial_failure(trial, traceback.format_exc())
File "C:\Users\ali\miniconda3\envs\testEnv\lib\site-packages\ray\tune\trial_runner.py", line 826, in _process_trial_failure
self._search_alg.on_trial_complete(trial.trial_id, error=True)
File "C:\Users\ali\miniconda3\envs\testEnv\lib\site-packages\ray\tune\suggest\search_generator.py", line 129, in on_trial_complete
self.searcher.on_trial_complete(
File "C:\Users\ali\miniconda3\envs\testEnv\lib\site-packages\ray\tune\suggest\suggestion.py", line 382, in on_trial_complete
self.searcher.on_trial_complete(
File "C:\Users\ali\miniconda3\envs\testEnv\lib\site-packages\ray\tune\suggest\optuna.py", line 222, in on_trial_complete
val = result.get(self.metric, None)
AttributeError: 'NoneType' object has no attribute 'get'