Given some constraints in my work, I need to benchmark in my (multi-agent RL) code against tabular Q learning/TD learning. I could not find an implementation of it in RLlib. Is there an easy way to get tabular Q-learning, e.g., by setting the DQN config appropriately? If there is no easy way to get it, I’d be happy to help create one!
1 Like
Hey inraos,
All our Q-Learning implementations are based on ANNs as Q function approximators.
A tabular Q function is not implemented. This is currently not on our Readmap and not something that people are usually looking for in a distributed RL framework. But it would be a very cool example I think!
@sven1977 has to decide whether he sees it fit.
I have a tabular Q learning algorithm in a framework that supports multi-agent games. There will be a little bit of work needed to get it working for your game. Message me if you’re interested in trying it out.
How can I DM on this platform?
Hi Indraos! 
Welcome to Ray community discuss! You might like to join the #rllib channel on slack too, for chatting.
You can reply to someone by clicking their name, then click Message.
Thanks,
Christy
@rusu24edward I’d be interested in this as well! Sometimes it would be useful to check DQN against tabular Q-learning as a sanity check on small instances. Is yours implemented as an RLlib Trainer/Algorithm? And is it publicly available anywhere?
@mgerstgrasser I currently have Q-learning as a part of Abmarl. It’s a bit outdated, and I have plans to update it to work with the newest changes. Take a look, I’m happy to answer any questions you have about it.
1 Like