Hello everybody, I’ve set up a custom multiagent config where I have two policies sharing most parts of a NN model (just input/output layers are individual). There are two agents [1, 2] each using one of the policies [hoist1, hoist2]. After first training step (train_batch_size: 256) I get the foll…

@arturn I’ve found the problem causing this error. For purposes of testing I set config variable train_batch_size to 128 and leave PPO config variable sgd_minibatch_size as default (128). Thus, sample batches of each agent contain less samples than sgd_minibatch_size, i.e. <128 and in method do_min…

AssertionError in ppo.py: KL is None, learner stats of at least one policy are empty

RLlib

arturn July 14, 2021, 3:13pm 2

Do you use a custom execution plan?
Is this ralated?

Topic		Replies	Views
Example code failed---multi_agent_two_trainers.py RLlib	0	140	March 20, 2024
Get_policy error when get an action from restored trained model- New API stack	12	59	April 22, 2025
AttributeError: 'SingleAgentEnvRunner' object has no attribute 'get_policy'	0	41	April 15, 2025
Help with ppo config in multiagent env with complex observations Configure Algorithm, Training, Evaluation, Scaling	0	18	April 11, 2025
RayTaskError(AttributeError) : ray::RolloutWorker.par_iter_next() RLlib	12	1403	February 21, 2022

AssertionError in ppo.py: KL is None, learner stats of at least one policy are empty

Related topics