Best way to use rllib with player vs player games

I asked a similar question a couple weeks ago: Best training algo for turn based board game?

Check out also what I’ve done so far at github.com/aronsar/domray