# \[Rllib\] compute\_single\_action() with an LSTM-PPO trainer fails
**URL:** https://discuss.ray.io/t/rllib-compute-single-action-with-an-lstm-ppo-trainer-fails/9234
**Category:** RLlib
**Created:** [February 3, 2023, 1:18pm UTC](https://discuss.ray.io/t/rllib-compute-single-action-with-an-lstm-ppo-trainer-fails/9234 "2023-02-03T13:18:22Z")
**Posts on this page:** 2
**Page:** 1
### Author:  [@Mirakolix\_Gallier](https://discuss.ray.io/u/Mirakolix_Gallier)
#### Post date: [February 3, 2023, 1:18pm UTC](https://discuss.ray.io/t/rllib-compute-single-action-with-an-lstm-ppo-trainer-fails/9234/1 "2023-02-03T13:18:22Z")
**How severe does this issue affect your experience of using Ray?**
- High: It blocks me to complete my task.
Python 3.10.8
ray==2.2.0
tensorflow==2.11.0
I am trying to test my trained agent using compute single action. The PPO default model with an LSTM wrapper: max\_sequence\_length = 10, cell\_size = 128. I am using “tf” as framework. The same PPO trainer without LSTM works fine.
trainer = trainer.restore(checkpoint) → works!
trainer.train() → works
trainer.compute\_single\_action(obs) → Leads to an error:
## Error Message:
2023-02-03 13:15:49,103 ERROR tf\_run\_builder.py:50 – Error fetching: [\, \, \, {‘action\_prob’: \, ‘action\_logp’: \, ‘action\_dist\_inputs’: \, ‘vf\_preds’: \}], feed\_dict={\: array([[4.97777018e-01, 2.29558937e-02, 1.00000000e+00, 4.97349521e-01,
1.52695383e-01, 4.19409695e-01, 4.96922025e-01, 2.58189322e-01,
2.04762132e-01, 4.96494528e-01, 2.63863812e-01, 2.46728897e-01,
4.96067031e-01, 1.77714728e-01, 3.25536652e-01, 4.98632011e-01,
2.55352076e-02, 1.00000000e+00, 4.99059508e-01, 2.91462471e-02,
4.44290551e-01, 4.99487004e-01, 1.55016766e-01, 4.97798038e-01,
4.99914501e-01, 1.94480268e-01, 3.41411342e-01, 5.00341997e-01,
2.42971370e-01, 2.72856084e-01, 1.00000000e+00, 3.48250044e-02,
4.98204514e-01, 4.73000000e-01, 4.66356019e-01, 4.33725122e-03,
1.46148357e-02, 1.00000000e+00, 1.00000000e-17, 1.00000000e-17]]), \: False, \: 2717636}
Traceback (most recent call last):
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/tensorflow/python/client/session.py”, line 1378, in \_do\_call
return fn(\*args)
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/tensorflow/python/client/session.py”, line 1361, in \_run\_fn
return self.\_call\_tf\_sessionrun(options, feed\_dict, fetch\_list,
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/tensorflow/python/client/session.py”, line 1454, in \_call\_tf\_sessionrun
return tf\_session.TF\_SessionRun\_wrapper(self.\_session, options, feed\_dict,
tensorflow.python.framework.errors\_impl.InvalidArgumentError: 2 root error(s) found.
(0) INVALID\_ARGUMENT: You must feed a value for placeholder tensor ‘default\_policy/seq\_lens’ with dtype int32 and shape [?]
[[{{node default\_policy/seq\_lens}}]]
[[default\_policy/model\_1/lstm/TensorArrayUnstack/strided\_slice/\_529]]
(1) INVALID\_ARGUMENT: You must feed a value for placeholder tensor ‘default\_policy/seq\_lens’ with dtype int32 and shape [?]
[[{{node default\_policy/seq\_lens}}]]
0 successful operations.
0 derived errors ignored.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/utils/tf\_run\_builder.py”, line 42, in get
self.\_executed = \_run\_timeline(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/utils/tf\_run\_builder.py”, line 102, in \_run\_timeline
fetches = sess.run(ops, feed\_dict=feed\_dict)
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/tensorflow/python/client/session.py”, line 968, in run
result = self.\_run(None, fetches, feed\_dict, options\_ptr,
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/tensorflow/python/client/session.py”, line 1191, in \_run
results = self.\_do\_run(handle, final\_targets, final\_fetches,
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/tensorflow/python/client/session.py”, line 1371, in \_do\_run
return self.\_do\_call(\_run\_fn, feeds, fetches, targets, options,
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/tensorflow/python/client/session.py”, line 1397, in \_do\_call
raise type(e)(node\_def, op, message) # pylint: disable=no-value-for-parameter
tensorflow.python.framework.errors\_impl.InvalidArgumentError: Graph execution error:
Detected at node ‘default\_policy/seq\_lens’ defined at (most recent call last):
File “/opt/conda/envs/tfenv/lib/python3.10/runpy.py”, line 196, in \_run\_module\_as\_main
return \_run\_code(code, main\_globals, None,
File “/opt/conda/envs/tfenv/lib/python3.10/runpy.py”, line 86, in \_run\_code
exec(code, run\_globals)
File “/home/jovyan/Level-3-Backtest-Engine-RL/reinforcement\_learning/test\_loops/solve\_lstm.py”, line 147, in
trained\_strategy = PPOTrainer(config=base\_config)
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/algorithms/algorithm.py”, line 441, in **init**
super(). **init** (
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/tune/trainable/trainable.py”, line 169, in **init**
self.setup(copy.deepcopy(self.config))
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/algorithms/algorithm.py”, line 566, in setup
self.workers = WorkerSet(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/evaluation/worker\_set.py”, line 169, in **init**
self.\_setup(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/evaluation/worker\_set.py”, line 259, in \_setup
self.\_local\_worker = self.\_make\_worker(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/evaluation/worker\_set.py”, line 941, in \_make\_worker
worker = cls(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/evaluation/rollout\_worker.py”, line 712, in **init**
self.\_build\_policy\_map(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/evaluation/rollout\_worker.py”, line 1970, in \_build\_policy\_map
self.policy\_map.create\_policy(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/policy/policy\_map.py”, line 146, in create\_policy
policy = create\_policy\_for\_framework(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/utils/policy.py”, line 117, in create\_policy\_for\_framework
return policy\_class(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/algorithms/ppo/ppo\_tf\_policy.py”, line 83, in **init**
base. **init** (
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/policy/dynamic\_tf\_policy\_v2.py”, line 87, in **init**
self.\_init\_state\_inputs(existing\_inputs)
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/policy/dynamic\_tf\_policy\_v2.py”, line 450, in \_init\_state\_inputs
self.\_seq\_lens = tf1.placeholder(
Node: ‘default\_policy/seq\_lens’
Detected at node ‘default\_policy/seq\_lens’ defined at (most recent call last):
File “/opt/conda/envs/tfenv/lib/python3.10/runpy.py”, line 196, in \_run\_module\_as\_main
return \_run\_code(code, main\_globals, None,
File “/opt/conda/envs/tfenv/lib/python3.10/runpy.py”, line 86, in \_run\_code
exec(code, run\_globals)
File “/home/jovyan/Level-3-Backtest-Engine-RL/reinforcement\_learning/test\_loops/solve\_lstm.py”, line 147, in
trained\_strategy = PPOTrainer(config=base\_config)
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/algorithms/algorithm.py”, line 441, in **init**
super(). **init** (
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/tune/trainable/trainable.py”, line 169, in **init**
self.setup(copy.deepcopy(self.config))
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/algorithms/algorithm.py”, line 566, in setup
self.workers = WorkerSet(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/evaluation/worker\_set.py”, line 169, in **init**
self.\_setup(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/evaluation/worker\_set.py”, line 259, in \_setup
self.\_local\_worker = self.\_make\_worker(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/evaluation/worker\_set.py”, line 941, in \_make\_worker
worker = cls(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/evaluation/rollout\_worker.py”, line 712, in **init**
self.\_build\_policy\_map(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/evaluation/rollout\_worker.py”, line 1970, in \_build\_policy\_map
self.policy\_map.create\_policy(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/policy/policy\_map.py”, line 146, in create\_policy
policy = create\_policy\_for\_framework(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/utils/policy.py”, line 117, in create\_policy\_for\_framework
return policy\_class(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/algorithms/ppo/ppo\_tf\_policy.py”, line 83, in **init**
base. **init** (
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/policy/dynamic\_tf\_policy\_v2.py”, line 87, in **init**
self.\_init\_state\_inputs(existing\_inputs)
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/policy/dynamic\_tf\_policy\_v2.py”, line 450, in \_init\_state\_inputs
self.\_seq\_lens = tf1.placeholder(
Node: ‘default\_policy/seq\_lens’
2 root error(s) found.
(0) INVALID\_ARGUMENT: You must feed a value for placeholder tensor ‘default\_policy/seq\_lens’ with dtype int32 and shape [?]
[[{{node default\_policy/seq\_lens}}]]
[[default\_policy/model\_1/lstm/TensorArrayUnstack/strided\_slice/\_529]]
(1) INVALID\_ARGUMENT: You must feed a value for placeholder tensor ‘default\_policy/seq\_lens’ with dtype int32 and shape [?]
[[{{node default\_policy/seq\_lens}}]]
0 successful operations.
0 derived errors ignored.
Original stack trace for ‘default\_policy/seq\_lens’:
File “/opt/conda/envs/tfenv/lib/python3.10/runpy.py”, line 196, in \_run\_module\_as\_main
return \_run\_code(code, main\_globals, None,
File “/opt/conda/envs/tfenv/lib/python3.10/runpy.py”, line 86, in \_run\_code
exec(code, run\_globals)
File “/home/jovyan/Level-3-Backtest-Engine-RL/reinforcement\_learning/test\_loops/solve\_lstm.py”, line 147, in
trained\_strategy = PPOTrainer(config=base\_config)
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/algorithms/algorithm.py”, line 441, in **init**
super(). **init** (
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/tune/trainable/trainable.py”, line 169, in **init**
self.setup(copy.deepcopy(self.config))
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/algorithms/algorithm.py”, line 566, in setup
self.workers = WorkerSet(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/evaluation/worker\_set.py”, line 169, in **init**
self.\_setup(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/evaluation/worker\_set.py”, line 259, in \_setup
self.\_local\_worker = self.\_make\_worker(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/evaluation/worker\_set.py”, line 941, in \_make\_worker
worker = cls(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/evaluation/rollout\_worker.py”, line 712, in **init**
self.\_build\_policy\_map(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/evaluation/rollout\_worker.py”, line 1970, in \_build\_policy\_map
self.policy\_map.create\_policy(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/policy/policy\_map.py”, line 146, in create\_policy
policy = create\_policy\_for\_framework(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/utils/policy.py”, line 117, in create\_policy\_for\_framework
return policy\_class(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/algorithms/ppo/ppo\_tf\_policy.py”, line 83, in **init**
base. **init** (
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/policy/dynamic\_tf\_policy\_v2.py”, line 87, in **init**
self.\_init\_state\_inputs(existing\_inputs)
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/policy/dynamic\_tf\_policy\_v2.py”, line 450, in \_init\_state\_inputs
self.\_seq\_lens = tf1.placeholder(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/tensorflow/python/ops/array\_ops.py”, line 3343, in placeholder
return gen\_array\_ops.placeholder(dtype=dtype, shape=shape, name=name)
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/tensorflow/python/ops/gen\_array\_ops.py”, line 6898, in placeholder
\_, \_, \_op, \_outputs = \_op\_def\_library.\_apply\_op\_helper(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/tensorflow/python/framework/op\_def\_library.py”, line 795, in \_apply\_op\_helper
op = g.\_create\_op\_internal(op\_type\_name, inputs, dtypes=None,
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/tensorflow/python/framework/ops.py”, line 3798, in \_create\_op\_internal
ret = Operation(
Traceback (most recent call last):
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/tensorflow/python/client/session.py”, line 1378, in \_do\_call
return fn(\*args)
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/tensorflow/python/client/session.py”, line 1361, in \_run\_fn
return self.\_call\_tf\_sessionrun(options, feed\_dict, fetch\_list,
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/tensorflow/python/client/session.py”, line 1454, in \_call\_tf\_sessionrun
return tf\_session.TF\_SessionRun\_wrapper(self.\_session, options, feed\_dict,
tensorflow.python.framework.errors\_impl.InvalidArgumentError: 2 root error(s) found.
(0) INVALID\_ARGUMENT: You must feed a value for placeholder tensor ‘default\_policy/seq\_lens’ with dtype int32 and shape [?]
[[{{node default\_policy/seq\_lens}}]]
[[default\_policy/model\_1/lstm/TensorArrayUnstack/strided\_slice/\_529]]
(1) INVALID\_ARGUMENT: You must feed a value for placeholder tensor ‘default\_policy/seq\_lens’ with dtype int32 and shape [?]
[[{{node default\_policy/seq\_lens}}]]
0 successful operations.
0 derived errors ignored.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File “/opt/conda/envs/tfenv/lib/python3.10/runpy.py”, line 196, in \_run\_module\_as\_main
return \_run\_code(code, main\_globals, None,
File “/opt/conda/envs/tfenv/lib/python3.10/runpy.py”, line 86, in \_run\_code
exec(code, run\_globals)
File “/home/jovyan/Level-3-Backtest-Engine-RL/reinforcement\_learning/test\_loops/solve\_lstm.py”, line 186, in
action = trained\_strategy.compute\_single\_action(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/algorithms/algorithm.py”, line 1495, in compute\_single\_action
action, state, extra = policy.compute\_single\_action(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/policy/policy.py”, line 466, in compute\_single\_action
out = self.compute\_actions\_from\_input\_dict(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/policy/tf\_policy.py”, line 326, in compute\_actions\_from\_input\_dict
fetched = builder.get(to\_fetch)
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/utils/tf\_run\_builder.py”, line 55, in get
raise e
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/utils/tf\_run\_builder.py”, line 42, in get
self.\_executed = \_run\_timeline(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/utils/tf\_run\_builder.py”, line 102, in \_run\_timeline
fetches = sess.run(ops, feed\_dict=feed\_dict)
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/tensorflow/python/client/session.py”, line 968, in run
result = self.\_run(None, fetches, feed\_dict, options\_ptr,
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/tensorflow/python/client/session.py”, line 1191, in \_run
results = self.\_do\_run(handle, final\_targets, final\_fetches,
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/tensorflow/python/client/session.py”, line 1371, in \_do\_run
return self.\_do\_call(\_run\_fn, feeds, fetches, targets, options,
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/tensorflow/python/client/session.py”, line 1397, in \_do\_call
raise type(e)(node\_def, op, message) # pylint: disable=no-value-for-parameter
tensorflow.python.framework.errors\_impl.InvalidArgumentError: Graph execution error:
Detected at node ‘default\_policy/seq\_lens’ defined at (most recent call last):
File “/opt/conda/envs/tfenv/lib/python3.10/runpy.py”, line 196, in \_run\_module\_as\_main
return \_run\_code(code, main\_globals, None,
File “/opt/conda/envs/tfenv/lib/python3.10/runpy.py”, line 86, in \_run\_code
exec(code, run\_globals)
File “/home/jovyan/Level-3-Backtest-Engine-RL/reinforcement\_learning/test\_loops/solve\_lstm.py”, line 147, in
trained\_strategy = PPOTrainer(config=base\_config)
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/algorithms/algorithm.py”, line 441, in **init**
super(). **init** (
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/tune/trainable/trainable.py”, line 169, in **init**
self.setup(copy.deepcopy(self.config))
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/algorithms/algorithm.py”, line 566, in setup
self.workers = WorkerSet(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/evaluation/worker\_set.py”, line 169, in **init**
self.\_setup(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/evaluation/worker\_set.py”, line 259, in \_setup
self.\_local\_worker = self.\_make\_worker(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/evaluation/worker\_set.py”, line 941, in \_make\_worker
worker = cls(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/evaluation/rollout\_worker.py”, line 712, in **init**
self.\_build\_policy\_map(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/evaluation/rollout\_worker.py”, line 1970, in \_build\_policy\_map
self.policy\_map.create\_policy(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/policy/policy\_map.py”, line 146, in create\_policy
policy = create\_policy\_for\_framework(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/utils/policy.py”, line 117, in create\_policy\_for\_framework
return policy\_class(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/algorithms/ppo/ppo\_tf\_policy.py”, line 83, in **init**
base. **init** (
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/policy/dynamic\_tf\_policy\_v2.py”, line 87, in **init**
self.\_init\_state\_inputs(existing\_inputs)
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/policy/dynamic\_tf\_policy\_v2.py”, line 450, in \_init\_state\_inputs
self.\_seq\_lens = tf1.placeholder(
Node: ‘default\_policy/seq\_lens’
Detected at node ‘default\_policy/seq\_lens’ defined at (most recent call last):
File “/opt/conda/envs/tfenv/lib/python3.10/runpy.py”, line 196, in \_run\_module\_as\_main
return \_run\_code(code, main\_globals, None,
File “/opt/conda/envs/tfenv/lib/python3.10/runpy.py”, line 86, in \_run\_code
exec(code, run\_globals)
File “/home/jovyan/Level-3-Backtest-Engine-RL/reinforcement\_learning/test\_loops/solve\_lstm.py”, line 147, in
trained\_strategy = PPOTrainer(config=base\_config)
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/algorithms/algorithm.py”, line 441, in **init**
super(). **init** (
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/tune/trainable/trainable.py”, line 169, in **init**
self.setup(copy.deepcopy(self.config))
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/algorithms/algorithm.py”, line 566, in setup
self.workers = WorkerSet(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/evaluation/worker\_set.py”, line 169, in **init**
self.\_setup(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/evaluation/worker\_set.py”, line 259, in \_setup
self.\_local\_worker = self.\_make\_worker(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/evaluation/worker\_set.py”, line 941, in \_make\_worker
worker = cls(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/evaluation/rollout\_worker.py”, line 712, in **init**
self.\_build\_policy\_map(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/evaluation/rollout\_worker.py”, line 1970, in \_build\_policy\_map
self.policy\_map.create\_policy(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/policy/policy\_map.py”, line 146, in create\_policy
policy = create\_policy\_for\_framework(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/utils/policy.py”, line 117, in create\_policy\_for\_framework
return policy\_class(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/algorithms/ppo/ppo\_tf\_policy.py”, line 83, in **init**
base. **init** (
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/policy/dynamic\_tf\_policy\_v2.py”, line 87, in **init**
self.\_init\_state\_inputs(existing\_inputs)
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/policy/dynamic\_tf\_policy\_v2.py”, line 450, in \_init\_state\_inputs
self.\_seq\_lens = tf1.placeholder(
Node: ‘default\_policy/seq\_lens’
2 root error(s) found.
(0) INVALID\_ARGUMENT: You must feed a value for placeholder tensor ‘default\_policy/seq\_lens’ with dtype int32 and shape [?]
[[{{node default\_policy/seq\_lens}}]]
[[default\_policy/model\_1/lstm/TensorArrayUnstack/strided\_slice/\_529]]
(1) INVALID\_ARGUMENT: You must feed a value for placeholder tensor ‘default\_policy/seq\_lens’ with dtype int32 and shape [?]
[[{{node default\_policy/seq\_lens}}]]
0 successful operations.
0 derived errors ignored.
Original stack trace for ‘default\_policy/seq\_lens’:
File “/opt/conda/envs/tfenv/lib/python3.10/runpy.py”, line 196, in \_run\_module\_as\_main
return \_run\_code(code, main\_globals, None,
File “/opt/conda/envs/tfenv/lib/python3.10/runpy.py”, line 86, in \_run\_code
exec(code, run\_globals)
File “/home/jovyan/Level-3-Backtest-Engine-RL/reinforcement\_learning/test\_loops/solve\_lstm.py”, line 147, in
trained\_strategy = PPOTrainer(config=base\_config)
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/algorithms/algorithm.py”, line 441, in **init**
super(). **init** (
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/tune/trainable/trainable.py”, line 169, in **init**
self.setup(copy.deepcopy(self.config))
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/algorithms/algorithm.py”, line 566, in setup
self.workers = WorkerSet(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/evaluation/worker\_set.py”, line 169, in **init**
self.\_setup(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/evaluation/worker\_set.py”, line 259, in \_setup
self.\_local\_worker = self.\_make\_worker(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/evaluation/worker\_set.py”, line 941, in \_make\_worker
worker = cls(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/evaluation/rollout\_worker.py”, line 712, in **init**
self.\_build\_policy\_map(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/evaluation/rollout\_worker.py”, line 1970, in \_build\_policy\_map
self.policy\_map.create\_policy(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/policy/policy\_map.py”, line 146, in create\_policy
policy = create\_policy\_for\_framework(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/utils/policy.py”, line 117, in create\_policy\_for\_framework
return policy\_class(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/algorithms/ppo/ppo\_tf\_policy.py”, line 83, in **init**
base. **init** (
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/policy/dynamic\_tf\_policy\_v2.py”, line 87, in **init**
self.\_init\_state\_inputs(existing\_inputs)
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/ray/rllib/policy/dynamic\_tf\_policy\_v2.py”, line 450, in \_init\_state\_inputs
self.\_seq\_lens = tf1.placeholder(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/tensorflow/python/ops/array\_ops.py”, line 3343, in placeholder
return gen\_array\_ops.placeholder(dtype=dtype, shape=shape, name=name)
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/tensorflow/python/ops/gen\_array\_ops.py”, line 6898, in placeholder
\_, \_, \_op, \_outputs = \_op\_def\_library.\_apply\_op\_helper(
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/tensorflow/python/framework/op\_def\_library.py”, line 795, in \_apply\_op\_helper
op = g.\_create\_op\_internal(op\_type\_name, inputs, dtypes=None,
File “/opt/conda/envs/tfenv/lib/python3.10/site-packages/tensorflow/python/framework/ops.py”, line 3798, in \_create\_op\_internal
ret = Operation(
---
### Author:  [@mannyv](https://discuss.ray.io/u/mannyv)
#### Post date: [February 3, 2023, 2:20pm UTC](https://discuss.ray.io/t/rllib-compute-single-action-with-an-lstm-ppo-trainer-fails/9234/2 "2023-02-03T14:20:54Z")
Hi @Mirakolix_Gallier,
Try this example:
>