Is there any way to pass additional parameter to custom action distribution

RLlib
1
  • Medium: It contributes to significant difficulty to complete my task, but I can work around it.

I would like to pass debug flag to custom action distribution so that I can print out message based on debug flag. Could someone shed me some light on it? Is there a way like custom_model_config to model?

Thanks,
James

2

Hi @James_Liu ,
No there is not.
Debugging RLlib usually works with calling ray.init(local_mode=True) before you start execution of any potentially distributed RLlib code. Then you can debug like you’d normally do and don’t have to do print debugging.

Cheers

1 Like
3

Hi @James_Liu,

As Arurn mentions there is not really a way to pass in configuration options directly. But, as a workaround, the action distribution init method takes in the model, so if you have a custom model you could use it to store configuration for the action distribution.

I am not sure if every use of an action distribution actually passes in the model though. Some uses may pass in or use the default value, None. You may want to check that first for the algorithms you are interested in.

Something like:

@override(ActionDistribution)
    def __init__(
        self,
        inputs: List[TensorType],
        model: TorchModelV2 = None,
        temperature: float = 1.0,
    ):
        # debug_action_dist does not exist in the built in rllib models
        # you would have to create a custom model that is a thin wrapper around the built in model
        # In the config you would put these in the config for the custom model
        # config: {"model": {"custom_model_config": {"action_dist_config": {"debug": True}}}}
        # In the custom model: self.debug_action_dist = model_config["custom_model_config"]["action_dist_config"]["debug"]
        self.debug = model.debug_action_dist

+1 for his suggestion of doing interactive debugging.

2 Likes
4

Both interactive debugging and piggyback model’s configuration are good suggestions for different scenarios. Thanks.
James