How to train a SAC agent with the offline API?

RLlib
1

Hi!

If I want to store training data for a SAC trainer with a SampleBatchBuilder, what data do I have to pass to each call to add values()?

2

Hey @trustee , thanks for posting this question. I think this example here would answer your question:

https://docs.ray.io/en/latest/rllib/rllib-offline.html#example-converting-external-experiences-to-batch-format

Once that’s done, you can train your SAC agent also with mixed input, like so:

config:
  input:
       [location (str) of your json output files from the SampleBatchBuilder example above]: 0.5,
       sampler: 0.5,

This would use the actual env (sampler) 50% of the time and 50% of the SAC training data would be taken from the json file. Just an example, you can set the ratios to different values, of course.