How to fix the error w.r.t flat.reshape?

RLlib
1

Hi everybody!
I got the error message
I don’t know what is the problem…
How to fix below the error?

  File "/home/bmil/anaconda3/envs/flow/lib/python3.7/site-packages/ray/tune/trial_runner.py", line 471, in _process_trial
    result = self.trial_executor.fetch_result(trial)
  File "/home/bmil/anaconda3/envs/flow/lib/python3.7/site-packages/ray/tune/ray_trial_executor.py", line 430, in fetch_result
    result = ray.get(trial_future[0], DEFAULT_GET_TIMEOUT)
  File "/home/bmil/anaconda3/envs/flow/lib/python3.7/site-packages/ray/worker.py", line 1538, in get
    raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(ValueError): ray::TD3.train() (pid=2401, ip=10.27.24.146)
  File "python/ray/_raylet.pyx", line 479, in ray._raylet.execute_task
  File "python/ray/_raylet.pyx", line 432, in ray._raylet.execute_task.function_executor
  File "/home/bmil/anaconda3/envs/flow/lib/python3.7/site-packages/ray/rllib/agents/trainer.py", line 522, in train
    raise e
  File "/home/bmil/anaconda3/envs/flow/lib/python3.7/site-packages/ray/rllib/agents/trainer.py", line 508, in train
    result = Trainable.train(self)
  File "/home/bmil/anaconda3/envs/flow/lib/python3.7/site-packages/ray/tune/trainable.py", line 332, in train
    result = self.step()
  File "/home/bmil/anaconda3/envs/flow/lib/python3.7/site-packages/ray/rllib/agents/trainer_template.py", line 110, in step
    res = next(self.train_exec_impl)
  File "/home/bmil/anaconda3/envs/flow/lib/python3.7/site-packages/ray/util/iter.py", line 758, in __next__
    return next(self.built_iterator)
  File "/home/bmil/anaconda3/envs/flow/lib/python3.7/site-packages/ray/util/iter.py", line 785, in apply_foreach
    for item in it:
  File "/home/bmil/anaconda3/envs/flow/lib/python3.7/site-packages/ray/util/iter.py", line 845, in apply_filter
    for item in it:
  File "/home/bmil/anaconda3/envs/flow/lib/python3.7/site-packages/ray/util/iter.py", line 845, in apply_filter
    for item in it:
  File "/home/bmil/anaconda3/envs/flow/lib/python3.7/site-packages/ray/util/iter.py", line 785, in apply_foreach
    for item in it:
  File "/home/bmil/anaconda3/envs/flow/lib/python3.7/site-packages/ray/util/iter.py", line 845, in apply_filter
    for item in it:
  File "/home/bmil/anaconda3/envs/flow/lib/python3.7/site-packages/ray/util/iter.py", line 1078, in build_union
    item = next(it)
  File "/home/bmil/anaconda3/envs/flow/lib/python3.7/site-packages/ray/util/iter.py", line 758, in __next__
    return next(self.built_iterator)
  File "/home/bmil/anaconda3/envs/flow/lib/python3.7/site-packages/ray/util/iter.py", line 785, in apply_foreach
    for item in it:
  File "/home/bmil/anaconda3/envs/flow/lib/python3.7/site-packages/ray/util/iter.py", line 785, in apply_foreach
    for item in it:
  File "/home/bmil/anaconda3/envs/flow/lib/python3.7/site-packages/ray/util/iter.py", line 785, in apply_foreach
    for item in it:
  File "/home/bmil/anaconda3/envs/flow/lib/python3.7/site-packages/ray/util/iter.py", line 793, in apply_foreach
    result = fn(item)
  File "/home/bmil/anaconda3/envs/flow/lib/python3.7/site-packages/ray/rllib/execution/rollout_ops.py", line 81, in <lambda>
    .for_each(lambda batches: SampleBatch.concat_samples(batches)) \
  File "/home/bmil/anaconda3/envs/flow/lib/python3.7/site-packages/ray/rllib/policy/sample_batch.py", line 93, in concat_samples
    out[k] = concat_aligned([s[k] for s in samples])
  File "/home/bmil/anaconda3/envs/flow/lib/python3.7/site-packages/ray/rllib/utils/memory.py", line 68, in concat_aligned
    output = flat.reshape(new_shape)
ValueError: cannot reshape array of size 388 into shape (30,14)
2

It’s my data type.

(pid=14431) tensor([[-1.0000,  0.0999,  0.0999, -1.0000,  0.0997,  0.1000,  0.0999, -1.0000,
(pid=14431)           0.0282,  0.0992, -1.0000, -0.1147, -0.2561,  0.0000]]) 
Shape of outputs: torch.Size([1, 14]) 
num_outputs:14

I reckon that this is the right shape, but Terminal put the error message below.
What is the problme?

  File "/home/bmil/anaconda3/envs/flow/lib/python3.7/site-packages/ray/rllib/agents/ddpg/ddpg_tf_policy.py", line 90, in get_distribution_inputs_and_class
    }, [], None)
  File "/home/bmil/anaconda3/envs/flow/lib/python3.7/site-packages/ray/rllib/models/modelv2.py", line 219, in __call__
    self.num_outputs, shape))

        outputs, state = res
        # print(outputs, outputs.shape, self.num_outputs)

        try:
            shape = outputs.shape
        except AttributeError:
            raise ValueError("Output is not a tensor: {}".format(outputs))
        else:
            if len(shape) != 2 or int(shape[1]) != self.num_outputs:
                raise ValueError(
                    "Expected output shape of [None, {}], got {}".format(
                        self.num_outputs, shape))
        if not isinstance(state, list):
            raise ValueError("State output is not a list: {}".format(state))

        self._last_output = outputs
        return outputs, state
3

Hey @Xim_Lee , could you post a short reproduction script here so I can debug your issue? Thanks! :slight_smile: