Can we add parameter noise instead of action noise to the Gradient-based algorithms?

carlorop · March 29, 2022, 10:42am

In general, parameter noise works better than action noise. Can we add parameter noise instead of action noise to the Gradient-based algorithms?

avnishn · March 31, 2022, 12:56am

I think that you can actually already do this with some algorithms, see this test:

github.com

ray-project/ray/blob/7f1bacc7dc9caf6d0ec042e39499bbf1d9a7d065/rllib/utils/exploration/tests/test_parameter_noise.py#L37

      
        
            
            
def test_dqn_parameter_noise(self):
                self.do_test_parameter_noise_exploration(
                    dqn.DQNTrainer,
                    dqn.DEFAULT_CONFIG,
                    "FrozenLake-v1",
                    {"is_slippery": False, "map_name": "4x4"},
                    np.array(0),
                )
            
            
def do_test_parameter_noise_exploration(
                self, trainer_cls, config, env, env_config, obs
            ):
                """Tests, whether an Agent works with ParameterNoise."""
                core_config = config.copy()
                core_config["num_workers"] = 0  # Run locally.
                core_config["env_config"] = env_config
            
            
    for fw in framework_iterator(core_config):
                    config = core_config.copy()

If something like this doesn’t work for you, can you open a feature request on github?

Topic		Replies	Views
Parameter noise exploration and policy gradient / actor critic RLlib	0	242	September 27, 2022
How does StochasticSampling work? RLlib	4	795	June 27, 2022
Reproducing results from stablebaselines 3 RLlib	2	547	August 6, 2021
Issue with custom environment RLlib	2	690	July 27, 2021
Entropy Regularization in PG? RLlib	9	656	September 17, 2022

Can we add parameter noise instead of action noise to the Gradient-based algorithms?

Related Topics