Question about features of environment

RLlib
1

Hello, how can I implement a fully connected NN to extract features from my environment and the learned latent vector are included as the part of observation?
I want to train two NNs together while the another NN is policy in DQN.

Thanks.

1 Like
2

You can do this in the Model interface. A CNN encoder can take an observation as an input and output a latent vector. This vector can be appended to the observation vector in the computation graph to input the Policy or Value network.

Example: ray/dreamer_model.py at master · ray-project/ray · GitHub

1 Like
3

Thanks. After I defined the custome encoder like a fully connected network (FCN) to extract features, can I use it in the step function in the env because I need to use the FCN to handle my observation which should be returned in the step function?

1 Like
4

Hey @Ethan , yeah, your env would need access to the model, which should be possible. You should be able get the RolloutWorker object that the env “sits in” from within the env’s e.g. constructor via:

from ray.rllib.evaluation.rollout_worker import get_global_worker

this_worker = get_global_worker()

Then, to get the model, you could do:

model = this_worker.policy_map["default_policy"].model