I want implement action masking for a simple environment with a dictionary observation space using the ApexDQN algorithm. The input_dict given to the forward method of my custom model sometimes is a SampleBatch and sometimes just a dictionary (with the exact same info the sample batch would have). …

Hello @user777888 , This post may be of some use for you. [image] [Contribution] [Help needed] Implementing easy action masking for distributional and dueling DQN RLlib How severe does this issue affect your experience of using Ray? None: Just asking a question out of cu…

The issue I’m facing uniquely comes up for dictionary action spaces, so I’m not sure how the attempt above would help. The docs already provide an action masking example for a DQN, but it only works for simple state spaces (i.e. not a dict). Action masks are passed as if they are part of the observ…

Hi @user777888 , I am not sure why it is sometimes one and sometimes the other. If I had to venture a gues I would suspect it is based on whether the observation is coming during the rollout sample phase versus the replay buffer in the training phase. This is how the observations are flattened duri…

Input to TorchModelV2 forward method inconsistent,

RLlib Configure Algorithm, Training, Evaluation, Scaling

mannyv April 19, 2023, 10:40am 2

Hello @user777888,

This post may be of some use for you.

Topic		Replies	Views
Action masking & Dict observation space & 'avail_actions'? Configure Algorithm, Training, Evaluation, Scaling	1	1195	August 4, 2023
How to not flatten action mask with Dict observation RLlib	0	491	April 8, 2022
Apply preprocessor in custom model RLlib	19	2600	May 13, 2024
How to flatten space when action masking? RLlib	7	1735	September 1, 2023
How to use LSTM or Attention Network action masking with nested dict action space? RLlib	0	289	August 24, 2023

Input to TorchModelV2 forward method inconsistent,

Related topics