Hello everybody, I’ve set up a custom multiagent config where I have two policies sharing most parts of a NN model (just input/output layers are individual). There are two agents [1, 2] each using one of the policies [hoist1, hoist2]. After first training step (train_batch_size: 256) I get the foll…

@arturn I’ve found the problem causing this error. For purposes of testing I set config variable train_batch_size to 128 and leave PPO config variable sgd_minibatch_size as default (128). Thus, sample batches of each agent contain less samples than sgd_minibatch_size, i.e. <128 and in method do_min…

AssertionError in ppo.py: KL is None, learner stats of at least one policy are empty

RLlib

arturn July 14, 2021, 3:13pm 2

Do you use a custom execution plan?
Is this ralated?

Topic		Replies	Views
RLlib Multi-Agent/ReplayBuffer DQN/SAC Error: Agents with Different Observation Space Shapes Configure Algorithm, Training, Evaluation, Scaling	2	555	March 9, 2024
Multi agent checkpoints - KeyError: 'default_policy' RLlib	1	450	October 30, 2021
Model isn't changing throw config RLlib	1	193	April 26, 2022
Only one policy metrics reported in the two trainer workflow RLlib	5	569	July 14, 2021
Inference multi agent not working RLlib	6	642	January 13, 2022

AssertionError in ppo.py: KL is None, learner stats of at least one policy are empty

Related Topics