Freezing observation filter for a policy server

RLlib
1

I noticed that when a policy is trained with "observation_filter": "MeanStdFilter" the mean and standard deviations get updated also the policy gets served (ie by a policy server) whenever a call to ExternalEnv.get_action is performed.
I could reproduce it with Ray 1.6.0 and 1.13.0

Is there a way to freeze the filter so to get fully deterministic behavior?

2

Hi Antoine! :wave:t3: Welcome to the Ray community! Thank you for the question!

Would you like to ask your question in RLlib Office Hours? :writing_hand:t3: Just add discuss link to your question to this doc: RLlib Office Hours - Google Docs

Thanks! Hope to see you there!