Multiple observations including RNN's

[hossein836]

How severe does this issue affect your experience of using Ray?

• Medium: It contributes to significant difficulty to complete my task, but I can work around it.
  Hi, I have an environment with space like this:

        self.observation_space = spaces.Dict({
            'RNN':spaces.Box(low= -1 * 10**10,high = 10**10, shape = (RNN.shape[1],)),
            "CNN": spaces.Box(low=-1 * 10 ** 10, high=10 ** 10, shape=(9,lookback+1,lookback+1), dtype=np.float32)})

so based on my understandings on seq_lens a simple RNN case training is like this:
suppose we have : seq_lens = 5, batch size = 100 , mini batch size = 4 , mini batch iteration = 10 and shuffle = TRUE, then training is like this:

so my question is what should be training look like for my above observation space?
the model is like below:

Here, the problem is CNN part doesn’t need sequential learning but the LSTM part does.
how can I handle this with RLlib? should I subclass RecurrentNetwork for my whole model or just LSTM part?
thank you

[arturn]

Hey @hossein836,

The standard way to handle this is indeed to subclass RecurrentNetwork for the whole model.
Only that the CNN stub will not receive state.
There is an example of something very similar to what you are trying to achieve:
Have a look!

Cheers.

[hossein836]

many thanks @arturn . can I ask you to check my understanding about seq_lens and mini batch from the picture above?
another question that I have is: the whole inheriting RecurrentNetwork thing is making sequence right?
so if I do it myself in my env, can I ignore that?( for e.g I return a sequence every step like below)

[arturn]

Hi @hossein836 ,

The shape of the the sequenced batch depends on the view requirements!
Your model has a view_requirements property that dictates how a batch should look.
First step would be to write your model how you think everything should look and with your expected inputs. If dimensions don’t fit you can come back and specify special view requirements and dig deeper!

Cheers