daniel
July 22, 2021, 6:23pm
1
Where could I find the actual default NN model for RLib A3CPolicy, I check the code but only find the high-level abstract class that has some function like forward to be implemented, is there any example that I can follow to define the NN model for customized Policy?
mannyv
July 23, 2021, 1:14pm
2
Hi @daniel ,
Here is one example:
from ray.rllib.models.tf.tf_modelv2 import TFModelV2
from ray.rllib.models.tf.fcnet import FullyConnectedNetwork as TFFCNet
from ray.rllib.models.torch.torch_modelv2 import TorchModelV2
from ray.rllib.models.torch.fcnet import FullyConnectedNetwork as TorchFCNet
from ray.rllib.utils.framework import try_import_tf, try_import_torch
tf1, tf, tfv = try_import_tf()
torch, nn = try_import_torch()
class CustomTorchRPGModel(TorchModelV2, nn.Module):
"""Example of interpreting repeated observations."""
def __init__(self, obs_space, action_space, num_outputs, model_config,
name):
super().__init__(obs_space, action_space, num_outputs, model_config,
name)
nn.Module.__init__(self)
self.model = TorchFCNet(obs_space, action_space, num_outputs,
model_config, name)
show original
There are many more examples of things rllib can do in the example directory for you to look through.
There is also a walk through in the documentation:
https://docs.ray.io/en/master/rllib-models.html?highlight=custom%20model#custom-models-implementing-your-own-forward-logic
2 Likes