How severe does this issue affect your experience of using Ray?
- High: It blocks me to complete my task.
I am trying to train a multi-agent model with action masking based off of this example. However it seems that in this example, the num_outputs is the same size as the action space, which is not the case in my model, so I am unsure how to proceed.
Some things I have attempted:
- Ignoring the num_outputs parameter when building my neural network and making the final layer instead have the same number of neurons as the action space. Received the following error:
RuntimeError: mat1 and mat2 shapes cannot be multiplied (32x19 and 256x256)
- Tried to emulate the implementation shown here. I was a bit unsure on the avail_actions and how to generate them, and I received the same RuntimeError as the previous attempt.
- Changing num_outputs to be the same size as the action space. I was unable to determine where num_outputs is being set. I also didn’t see any options in the configs that appeared to refer to the num_outputs.
Here is the code for my model (simplified).
class MyModel(TorchModelV2): def __init__(self, obs_space, act_space, num_outputs, *args, **kwargs): TorchModelV2.__init__(self, obs_space, act_space, num_outputs, *args, **kwargs) self.model = nn.Sequential( (nn.Linear(flatdim(obs_space) - act_space.n,8192)), (nn.Linear(8192,num_outputs)), ) def forward(self, input_dict, state, seq_lens): assert torch.equal(input_dict["obs_flat"][:,-19:], input_dict["obs"]["action_mask"]) model_out = self.model(input_dict["obs_flat"][:,:-19]) return model_out, state
Any help would be greatly appreciated!