You are correct—your code sets "type": "MultiAgentPrioritizedReplayBuffer" as a string. However, the error occurs because somewhere in the config processing, the "type" field is being converted from a string to a class object (likely by RLlib’s internal validate_buffer_config or similar logic). This means that by the time the check if "EpisodeReplayBuffer" in config["replay_buffer_config"]["type"]: runs, "type" is no longer a string, but a class, causing the TypeError.
This is a known RLlib bug: after config validation, "type" is replaced with the class, but later code still assumes it’s a string and tries to use in on it. This issue is discussed in the Ray community and is not due to your code, but to RLlib internals. There is no user-side fix except to downgrade to a version where this bug does not occur, or to patch RLlib locally to check the type before using in (e.g., if isinstance(type_val, str) and "EpisodeReplayBuffer" in type_val:).
Would you like more detail on possible workarounds or a code patch?
Sources:
Hint: Mention @RunLLM in the post for followups.