Hello,
We’d like to use SAC with MAML so that we can incorporate some offline training. Is this fairly straightforward in RLLib? If not, can anyone suggest a minimal course of action to implement this?
Hello,
We’d like to use SAC with MAML so that we can incorporate some offline training. Is this fairly straightforward in RLLib? If not, can anyone suggest a minimal course of action to implement this?
Hi @lucas_spangher,
How are you imagining this training would go?
Like this?
Or like this?
Loop
You could also save offline data from MAML to use with CQL but there would be some bookkeeping to make sure the environments matched.
Also the RLLIB CQL implementation is very new so there may be some lingering issues that have not been found yet.