I am trying to use a customized action mask model. I also would like to customize the flatten observation before passing it into the model. So for obs_flat, I notice that RLlib has restore_original_dimensions to unpack the obs. What if I want to pack the obs? For example, I have obs with Repeated type and within the Repeated Type, it also has Dict Type. Instead of using obs_flat and manually figure out the dimension of each different feature in obs, I want to flatten the obs in my own needs.
‘package_skills’: Repeated(package_skills_space, max_len=self.capacity),
‘visible_skills’: Repeated(visible_skills_space, max_len=self.max_skill_in_view)
e.g. obs = {
‘Item1’: Repeated(Dict({‘position’: Box(-1,1,shape=(3,)), ‘value’: Discrete(5)})),
‘Item2’: Repeated(Dict({‘position’: Box(-1,1,shape=(3,)), ‘info’: Discrete(5)})),
}
I want to flatten Item1 and Item2 separately so that I can feed two items separately into the model.
I first try to use the preprocessor class to do it, but it seems only work with non-tensor type. What should I do with the tensor type obs and achieve my purpose?