# Unable to use Dict observation space **URL:** \ **Category:** RLlib\ **Created:** [March 13, 2022, 2:54pm UTC](https://discuss.ray.io/t/unable-to-use-dict-observation-space/5395 "2022-03-13T14:54:56Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![Loneknight73](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/loneknight73/32/2271_2.png) [@Loneknight73](https://discuss.ray.io/u/Loneknight73)\ **Post date:** [March 13, 2022, 2:54pm UTC](https://discuss.ray.io/t/unable-to-use-dict-observation-space/5395/1 "2022-03-13T14:54:56Z")
Hello, my goal is to train a NN for a card game with self-play; this is my first experience in this field (I am using ray-2.0.0.dev0). In my PettingZoo environment I have: ```auto self.action_spaces = {agent: Dict({ GameState.BIDDING: Discrete(BID_ACTIONS), GameState.CHOOSE_TRUMP: Discrete(CHOOSE_TRUMP_ACTIONS), GameState.TRICK: Discrete(TRICK_ACTIONS) }) for agent in self.agents} self.observation_spaces = {agent: Dict({ 'observations': Dict({ 'gamestate' : Discrete(len(GameState)), 'player_hand' : Box(low=0, high=1, shape=(TRICK_ACTIONS,), dtype=bool), # [...] other state I keep temporarily commented }), 'action_mask': Dict({ GameState.BIDDING: Box(low=0, high=1, shape=(BID_ACTIONS,), dtype=bool), GameState.CHOOSE_TRUMP: Box(low=0, high=1, shape=(CHOOSE_TRUMP_ACTIONS,), dtype=bool), GameState.TRICK: Box(low=0, high=1, shape=(TRICK_ACTIONS,), dtype=bool) }) }) for agent in self.agents} ``` To be able to apply action masks, as suggested in the documentation, I defined a custom model: ```auto class ParametricActionsModel(TFModelV2): def __init__ (self, obs_space, action_space, num_outputs, model_config, name, **kw): super(). __init__ ( obs_space, action_space, num_outputs, model_config, name, **kw) self.prep = get_preprocessor(obs_space.original_space.spaces['observations']) orig_obs_space = self.prep(obs_space.original_space.spaces['observations']) self.action_embed_model = FullyConnectedNetwork( orig_obs_space, action_space, num_outputs, model_config, name + "_action_embedding" ) def forward(self, input_dict, state, seq_lens): # Extract the available actions tensor from the observation. action_mask = input_dict["obs"]["action_mask"] # Compute the predicted action embedding orig_obs = self.prep.transform(input_dict["obs"]["observations"]) action_logits, _ = self.action_embed_model({ "obs": orig_obs }) # Mask out invalid actions (use tf.float32.min for stability) inf_mask = tf.maximum(tf.math.log(action_mask), tf.float32.min) return action_logits + inf_mask, state def value_function(self): return self.action_embed_model.value_function() ``` However, I get the error below: ```auto File "E:/Vmware shared folder/python/BriscolaChiamata/train.py", line 44, in forward orig_obs = self.prep.transform(input_dict["obs"]["observations"]) TypeError: transform() missing 1 required positional argument: 'observation' at time: 1.64718e+09 ``` The point is: the actual observation I want to feed the NN with is input\_dict[“obs”][“observations”], but I don’t know how to actually pass it to self.action\_embed\_model(). I am not even sure the `ParametricActionsModel. __init__ ()` does the right things. By the way, a previous temporary version of this code, where I had: ```auto self.observation_spaces = {agent: Dict({ 'observations': Box( #... 'action_mask': # as above ``` and the custom ParametricActionsModel (without the preprocessor and the transform()), did not cause errors. However, using a Dict space helps me to keep the code clearer. Can someone please help me? I did not found existing examples helpful enough for my (very low) level of expertise. Thanks in advance **How severe does this issue affect your experience of using Ray?** - High: It blocks me to complete my task. ---
**Author:** ![arturn](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/arturn/32/2096_2.png) [@arturn](https://discuss.ray.io/u/arturn)\ **Post date:** [March 13, 2022, 7:56pm UTC](https://discuss.ray.io/t/unable-to-use-dict-observation-space/5395/2 "2022-03-13T19:56:26Z")
Hi @Loneknight73 , > [@Loneknight73](#): > > ```auto > File "E:/Vmware shared folder/python/BriscolaChiamata/train.py", line 44, in forward > orig_obs = self.prep.transform(input_dict["obs"]["observati > > ``` I don’t get this error at the moment, since your preprocessor should only need one argument: the observation. But I noticed you are using `get_preprocessor()` instead of `get_preprocessor_for_space()`. Does that mess up your usage of the preprocessor? Also: Have a look at [this](https://docs.ray.io/en/latest/rllib/rllib-models.html#custom-preprocessors-and-environment-filters)! ---
**Author:** ![Loneknight73](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/loneknight73/32/2271_2.png) [@Loneknight73](https://discuss.ray.io/u/Loneknight73)\ **Post date:** [March 14, 2022, 6:38pm UTC](https://discuss.ray.io/t/unable-to-use-dict-observation-space/5395/3 "2022-03-14T18:38:53Z")
Hi @arturn , thanks for your answer. I could not try your suggestion, but I remember orig\_obs\_space having the right shape. About: > [@arturn](#): > > Also: Have a look at [this](https://docs.ray.io/en/latest/rllib/rllib-models.html#custom-preprocessors-and-environment-filters)! yes, I saw it. It seems to me custom preprocessors are deprecated, but builtin ones are not. It’s not clear if there are other automated ways to convert a dictionary into a suitable NN input.