Mixed Action Space - SAC

RLlib
1

1. Severity of the issue: (select one)
None: I’m just curious or want clarification.
Low: Annoying but doesn’t hinder my work.
Medium: Significantly affects my productivity but can find a workaround.
High: Completely blocks me.

2. Environment:

  • Ray version: 2.51.0

Are there any plans to support SAC with mixed action spaces e.g. discrete and continuous represented as a dict?

When I run this example - ray/rllib/examples/actions/nested_action_spaces.py at master · ray-project/ray · GitHub with SAC, I hit this UnsupportedSpaceException.

2

Currently, RLlib’s SAC does not support mixed action spaces (e.g., Dict or Tuple spaces combining discrete and continuous actions). SAC only supports single Box (continuous), Discrete, or Simplex action spaces, and will raise an UnsupportedSpaceException for other types, including Dict or Tuple spaces with mixed action types. This is enforced in the SAC policy code and is a known limitation (source, source).

There are no official indications in the provided sources that support for mixed action spaces in SAC is planned or under development. For now, you must use algorithms like PPO for such spaces, or flatten/encode your action space to fit within a supported type (source, source). Would you like more detail on possible workarounds or alternative approaches?

Sources:

Hint: Mention @RunLLM in the post for followups.