How to avoid the preprocess concatenating of obs when using RLModule

RLlib Configure Algorithm, Training, Evaluation, Scaling
1

I’m using a SingleAgentRLModuleSpec, my observation_space is a dict,but when i want to propcess it in CustomPPOTorchRLModule, batch[‘obs’] has been concacted automatically. How to avoid it?

spec = SingleAgentRLModuleSpec(
module_class=CustomPPOTorchRLModule,
observation_space= gym.spaces.Dict({“obs”: gym.spaces.Box(low=0, high=255, shape=(208, 416, 3), dtype=np.uint8), “aux”: gym.spaces.Box(low=-1.0, high=1.0, shape=(6,), dtype=np.float32)}),
action_space=gym.spaces.Discrete(5),
model_config_dict={
},
catalog_class=CustomPPOCatalog,
)

2

Hi! Have you tried config…experimental(_disable_preprocessor_api=True) ?