If I Initialize from a predefined model, I notice that there is an additional network for Value Function by default.
However ES/ARS never make use of the Value Function. So is this part of the model even needed or can it be safely deleted ?
If so then the original implementation in ray/es.py at master · ray-project/ray · GitHub should not update the parameters for the value function.
Possible script to run:
tune.run(ESTrainer, config={“env”: “CartPole-v0”,
“framework”: “torch”,
“num_workers”: 1,
“stepsize”: 0.01,
“model”: {
“fcnet_hiddens”: [],
}
},
stop={“training_iteration”: 10}
)