It seems likely that you can encode agent heterogeneity in discount rates by making the discount factor (gamma) an observable part of each agent’s observation space, and sampling it per agent per episode. RLlib does not natively support per-agent or per-episode discount rates out of the box, but you can include the discount rate as part of the observation and let the policy learn to condition on it. The actual discounting in RLlib is global (set in the config), so the environment and reward structure must be designed to reflect the agent’s discount preference, or you must implement custom reward processing or value targets if you want true per-agent discounting during training. See discussion in this thread and the multi-agent documentation.
Would you like more detail on how to implement this, or on possible workarounds for per-agent discounting?
Sources:
Hint: Mention @Herald in the post for followups.