Question about features of environment

Hello, how can I implement a fully connected NN to extract features from my environment and the learned latent vector are included as the part of observation?
I want to train two NNs together while the another NN is policy in DQN.


You can do this in the Model interface. A CNN encoder can take an observation as an input and output a latent vector. This vector can be appended to the observation vector in the computation graph to input the Policy or Value network.

Example: ray/ at master · ray-project/ray · GitHub

Thanks. After I defined the custome encoder like a fully connected network (FCN) to extract features, can I use it in the step function in the env because I need to use the FCN to handle my observation which should be returned in the step function?