# Ddpg restore problem **URL:** \ **Category:** RLlib\ **Created:** [March 16, 2022, 7:04am UTC](https://discuss.ray.io/t/ddpg-restore-problem/5420 "2022-03-16T07:04:29Z")\ **Posts on this page:** 1\ **Page:** 1
**Author:** ![qinmoelei](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/qinmoelei/32/2284_2.png) [@qinmoelei](https://discuss.ray.io/u/qinmoelei)\ **Post date:** [March 16, 2022, 7:04am UTC](https://discuss.ray.io/t/ddpg-restore-problem/5420/1 "2022-03-16T07:04:29Z")
I am new to the RLlib and is still learning how to use it. When I try to restore my pretrained DDPG model from the check point, it raise an error as below TypeError: \ is out of scope and cannot be used here. Use return values, explicit Python locals or TensorFlow collections to access it. Please see [أداء أفضل مع وظيفة tf  |  TensorFlow Core](https://www.tensorflow.org/guide/function#all_outputs_of_a_tffunction_must_be_return_values) for more information. \ was defined here: File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/runpy.py”, line 193, in \_run\_module\_as\_main “ **main** ”, mod\_spec) File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/runpy.py”, line 85, in \_run\_code exec(code, run\_globals) File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/ipykernel\_launcher.py”, line 16, in app.launch\_new\_instance() File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/traitlets/config/application.py”, line 846, in launch\_instance app.start() File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/ipykernel/kernelapp.py”, line 677, in start self.io\_loop.start() File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/tornado/platform/asyncio.py”, line 199, in start self.asyncio\_loop.run\_forever() File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/asyncio/base\_events.py”, line 541, in run\_forever self.\_run\_once() File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/asyncio/base\_events.py”, line 1786, in \_run\_once handle.\_run() File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/asyncio/events.py”, line 88, in \_run self.\_context.run(self.\_callback, \*self.\_args) File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/ipykernel/kernelbase.py”, line 471, in dispatch\_queue await self.process\_one() File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/ipykernel/kernelbase.py”, line 460, in process\_one await dispatch(\*args) File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/ipykernel/kernelbase.py”, line 367, in dispatch\_shell await result File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/ipykernel/kernelbase.py”, line 662, in execute\_request reply\_content = await reply\_content File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/ipykernel/ipkernel.py”, line 360, in do\_execute res = shell.run\_cell(code, store\_history=store\_history, silent=silent) File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/ipykernel/zmqshell.py”, line 532, in run\_cell return super().run\_cell(\*args, \*\*kwargs) File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/IPython/core/interactiveshell.py”, line 2915, in run\_cell raw\_cell, store\_history, silent, shell\_futures) File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/IPython/core/interactiveshell.py”, line 2960, in \_run\_cell return runner(coro) File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/IPython/core/async\_helpers.py”, line 78, in _pseudo\_sync\_runner coro.send(None) File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/IPython/core/interactiveshell.py”, line 3186, in run\_cell\_async interactivity=interactivity, compiler=compiler, result=result) File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/IPython/core/interactiveshell.py”, line 3377, in run\_ast\_nodes if (await self.run\_code(code, result, async_=asy)): File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/IPython/core/interactiveshell.py”, line 3457, in run\_code exec(code\_obj, self.user\_global\_ns, self.user\_ns) File “/tmp/ipykernel\_3631/4187836080.py”, line 12, in trainer=ddpg.DDPGTrainer(env=TrainEnv, config=model\_config) File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/ray/rllib/agents/trainer.py”, line 747, in **init** sync\_function\_tpl) File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/ray/tune/trainable.py”, line 124, in **init** self.setup(copy.deepcopy(self.config)) File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/ray/rllib/agents/trainer.py”, line 827, in setup num\_workers=self.config[“num\_workers”]) File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/ray/rllib/agents/trainer.py”, line 2002, in \_make\_workers logdir=self.logdir, File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/ray/rllib/evaluation/worker\_set.py”, line 132, in **init** spaces=spaces, File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/ray/rllib/evaluation/worker\_set.py”, line 540, in \_make\_worker spaces=spaces, File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/ray/rllib/evaluation/rollout\_worker.py”, line 590, in **init** seed=seed) File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/ray/rllib/evaluation/rollout\_worker.py”, line 1578, in \_build\_policy\_map conf, merged\_conf) File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/ray/rllib/policy/policy\_map.py”, line 134, in create\_policy observation\_space, action\_space, merged\_config) File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/ray/rllib/policy/tf\_policy\_template.py”, line 252, in **init** get\_batch\_divisibility\_req=get\_batch\_divisibility\_req, File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/ray/rllib/policy/dynamic\_tf\_policy.py”, line 271, in **init** self.exploration = self.\_create\_exploration() File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/ray/rllib/policy/policy.py”, line 755, in _create\_exploration self.config.get(“framework”, “tf”))) File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/ray/rllib/utils/from\_config.py”, line 192, in from\_config object_ = constructor(\*ctor\_args, \*\*ctor\_kwargs) File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/ray/rllib/utils/exploration/ornstein\_uhlenbeck\_noise.py”, line 70, in **init** device=None) File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/ray/rllib/utils/framework.py”, line 240, in get\_variable “shape”: shape File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/tensorflow/python/ops/variable\_scope.py”, line 1633, in get\_variable aggregation=aggregation) File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/tensorflow/python/ops/variable\_scope.py”, line 1349, in get\_variable aggregation=aggregation) File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/tensorflow/python/ops/variable\_scope.py”, line 604, in get\_variable aggregation=aggregation) File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/tensorflow/python/ops/variable\_scope.py”, line 556, in \_true\_getter aggregation=aggregation) File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/tensorflow/python/ops/variable\_scope.py”, line 974, in \_get\_single\_variable aggregation=aggregation) File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/tensorflow/python/util/traceback\_utils.py”, line 150, in error\_handler return fn(\*args, \*\*kwargs) File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/tensorflow/python/ops/variables.py”, line 265, in **call** return cls.\_variable\_v1\_call(\*args, \*\*kwargs) File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/tensorflow/python/ops/variables.py”, line 225, in \_variable\_v1\_call shape=shape) File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/tensorflow/python/ops/variables.py”, line 203, in previous\_getter = lambda \*\*kwargs: default\_variable\_creator(None, \*\*kwargs) File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/tensorflow/python/ops/variable\_scope.py”, line 2721, in default\_variable\_creator shape=shape) File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/tensorflow/python/util/traceback\_utils.py”, line 150, in error\_handler return fn(\*args, \*\*kwargs) File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/tensorflow/python/ops/variables.py”, line 269, in **call** return super(VariableMetaclass, cls). **call** (\*args, \*\*kwargs) File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/tensorflow/python/ops/resource\_variable\_ops.py”, line 1676, in **init** distribute\_strategy=distribute\_strategy) File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/tensorflow/python/ops/resource\_variable\_ops.py”, line 1832, in \_init\_from\_args graph\_mode=self.\_in\_graph\_mode) File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/tensorflow/python/ops/resource\_variable\_ops.py”, line 238, in eager\_safe\_variable\_handle graph\_mode, initial\_value) File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/tensorflow/python/ops/resource\_variable\_ops.py”, line 166, in \_variable\_handle\_from\_shape\_and\_dtype container=container) File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/tensorflow/python/ops/gen\_resource\_variable\_ops.py”, line 1215, in var\_handle\_op allowed\_devices=allowed\_devices, name=name) File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/tensorflow/python/framework/op\_def\_library.py”, line 742, in \_apply\_op\_helper attrs=attr\_protos, op\_def=op\_def) File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/tensorflow/python/framework/ops.py”, line 3784, in \_create\_op\_internal op\_def=op\_def) File “/home/qinmolei/anaconda3/envs/FinRL/lib/python3.7/site-packages/tensorflow/python/framework/ops.py”, line 2175, in **init** self.\_traceback = tf\_stack.extract\_stack\_for\_node(self.\_c\_op) The tensor \ cannot be accessed from here, because it was defined in \, which is out of scope. Can any one tell me where did I go wrong? Thanks