Trying around with the latest action masking example by @arturn, I was trying to feed it into my problem. I noticed that in my gym envr, some static features (static = they are fed into neural network but do not change along episode) were not part of the “real_obs” or “observations”.
Now I get following error message by PPO Config in ray 2.6.1 (nightly build), tensorflow 2.11
ValueError: No default encoder config for obs space=Box(-1.0, 1.0, (1, 164), float64), lstm=False and attention=False found. 2D Box spaces are not supported. They should be either flattened to a 1D Box space or enhanced to be a 3D box space.
It seems that this is related to the following setting .experimental(_disable_preprocessor_api=True)
What is the best way? Do a manual flattening with gym.spaces.flatten?
config.experimental(_disable_preprocessor_api=True) makes is so that you observations are not transformed. One standard transformation that RLlib does is to flatten nested observation spaces so that the model becomes less complex.
The observation space that you are looking at is not flattened. It is of size dimension (1, 164). Any chance you can just squeeze it?
Useful hints, @arturn . During first checks it looks like that the FlattenObservation Wrapper delivered by gymnasium is quite handy. Or is there any hidden functionality of ray which prohibits usage of that wrapper?
Is it possible to have a mask for the observation_space as well? Imagine a situation where the observation_space is a space.Dict with keys that change throughout the steps. The action_space is also a space.Dict with the same keys as the observation_space .