Normalizing Observations

[Username1]

Hi! yes, it is a matter of just applying this configuration:

"observation_filter": "MeanStdFilter",

I took the example from here:

How to correctly apply observation normalization?

I’m using MeanStdFilter in my PPO example. It works during the training process. But, i’m not sure whether to use filtered observations when calling trainer.compute_single_action. In my opinion, the model was trained based on filtered data, and thus the compute_single_action function should also take the filtered obs as the input. However, in my example, the action obtained without the filtered obs performs better than the action obtained with filtered obs. Then, what is the correct way to …

Here is the source code, to make sure it’s applying what you need:

ray-project/ray/blob/60d4d5e1aaa9fde3cf541ee335e284d05e75679c/rllib/utils/filter.py

And this forum is where I’ve got my answers from:

[rllib] Best practice for normalizing observations with running mean

Hope this heps! cheers.