Correct way to use APPO with 'use_kl_loss'

Hey all am trying to use APPO with ‘use_kl_loss’ being true and it always ends with error tracing to kl = fetches[pi_id][LEARNER_STATS_KEY].get("kl") where LEARNER_STATS_KEY isn’t found in the dict… has anyone ran into this issue or is it a bug?

1 Like