Hello,
can anyone explain me what is the reason of the following two lines:
# Push obs through "unwrapped" net's `forward()` first.
wrapped_out, _ = self._wrapped_forward(input_dict, [], None)
in the following file:
it is in the LSTMWrapper class and forward method.
Thanks.
Hi @mg64ve,
When you set config["model"]["use_lstm"] =True, rllib will automatically create an lstm for you. This is the code you are looking at.
The way it does it is it first creates a model (base) with the layers before the lstm. These are fullyconnected or conv layers. Then it creates a second model that has the lstm and output layers (logits and value).
That line you asked about is when it passes the inputs from the environment to the base model to get the embedding inputs for the lstm.
Ok Thank you very much @mannyv for your explanation.
I would like to implement a custom RNN model with a L2 weights regularization.
From this comment:
# === Built-in options ===
# FullyConnectedNetwork (tf and torch): rllib.models.tf|torch.fcnet.py
# These are used if no custom model is specified and the input space is 1D.
# Number of hidden layers to be used.
"fcnet_hiddens": [256, 256],
# Activation function descriptor.
# Supported values are: "tanh", "relu", "swish" (or "silu"),
# "linear" (or None).
"fcnet_activation": "tanh",
I believe for custom models there is no FC input layer, right?
Now, if I implement my custom model, I believe this would not have a previous action/reward stacking mechanism, right? any good hint on how to implement it?