It could be possible that RLlib’s MASAC (Multi-Agent Soft Actor-Critic) is based on the standard SAC algorithm, extended for multi-agent settings. Each agent has its own actor (policy network) and twin critics (Q-networks), and the entropy temperature (alpha) is learned to balance exploration and exploitation. The actor outputs actions, the critics estimate Q-values, and the temperature controls entropy regularization. See SACConfig and code and default SAC RLModule.
To extend or subclass actors, critics, or temperature modules, you might subclass the relevant RLModule or model classes. For example, you can subclass DefaultSACRLModule for custom actor/critic logic, or register a custom model for policy or Q-networks via policy_model_config and q_model_config in SACConfig. See default_sac_rl_module.py and custom model discussion.
Would you like a step-by-step example of subclassing one of these components?
Sources:
Hint: Mention @Herald in the post for followups.