@rusu24edward , if I switch to multi-agent setting, I have to change a lot of code in rest of my codebase that uses learned policy. I think I should first give a shot to exploration. Can you please look into my follow up question here Using different get_exploration_action method pre and post training ?