With the new API stack enabled and no custom RLModule or model config, RLlib does not automatically create separate CNN/MLP branches for Dict observation spaces. By default, RLlib will attempt to flatten the Dict into a 1D vector and process it with a single MLP, which is inefficient for image data and not what you want for your setup. This is confirmed by user reports and error messages in recent discussions and issues, e.g., “No default encoder config for obs space=Dict(…)” and the need for custom handling of Dict spaces in the new stack (source, source, source).
To properly process your Dict obs (pixels via CNN, features via MLP), you should implement a custom RLModule that splits the Dict and routes each part through the appropriate encoder, then concatenates the results before the policy/value heads. This is the recommended approach and is demonstrated in the examples you referenced (action_masking_rlm.py and tiny_atari_cnn_rlm.py). Would you like a step-by-step breakdown or a code template for this pattern?
Sources:
- How to use PPO with Dict observation space (pixels + features) in Ray 2.48.0?
- Using Dict observation space with custom RLModule
- RLlib GitHub Issue: Observation space with 2 dimensions not working with the new API stack
- custom_cnn_rl_module.py example
- action_masking_rlm.py
Hint: Mention @RunLLM in the post for followups.