How to make the A3C tutorial work?

RLlib
1

In the documentation, I found this tutorial on A3C. I really like the tutorial as it’s very simple and it covers asynchronous methods, but unfortunately it seems outdated and it uses LSTMPolicy, which is not present in the current version of RLLib. Is there a working version of the same or similar tutorial or maybe a fix that would make it work, such as replacing the LSTMPolicy with something else?

2

Hey @konstmish , great question. It does seem like the example is quite outdated and on the Ray core docs, not RLlib’s. Hmm, the algo works out of the box, though. You could get the results using an LSTM wrapped default model (in this case: a classic Atari Conv2D Stack) by doing:

rllib train --run A3C --env Pong-v0 --config {"model": {"use_lstm": true}}

3

Thanks for the suggestion. What you propose makes sense to get the results, but I actually wanted to tweak things, so I hoped there is a way to make the tutorial work.