# Initialize replay buffer

**URL:** <https://discuss.ray.io/t/initialize-replay-buffer/2706>\
**Category:** RLlib\
**Created:** [June 29, 2021, 6:42pm UTC](https://discuss.ray.io/t/initialize-replay-buffer/2706 "2021-06-29T18:42:54Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![vineet54](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/vineet54/32/809_2.png) [@vineet54](https://discuss.ray.io/u/vineet54)\
**Post date:** [June 29, 2021, 6:42pm UTC](https://discuss.ray.io/t/initialize-replay-buffer/2706/1 "2021-06-29T18:42:54Z")

</div>

Hello,

I’d like to initialize the replay buffer in SAC (or any other off-policy algo) with experience from a non-RL policy. I want the RL agent to start out learning from this policy. What is the best way to do this?

---

<div class="post-metadata">

**Author:** ![michaelzhiluo](https://avatars.discourse-cdn.com/v4/letter/m/41988e/32.png) [@michaelzhiluo](https://discuss.ray.io/u/michaelzhiluo)\
**Post date:** [July 1, 2021, 7:42am UTC](https://discuss.ray.io/t/initialize-replay-buffer/2706/2 "2021-07-01T07:42:44Z")

</div>

I did something similar for offline RL agent CQL: [ray/cql.py at master · ray-project/ray · GitHub](https://github.com/ray-project/ray/blob/master/rllib/agents/cql/cql.py#L131)

They key part is to add the dataset in the `after_init` function in the Trainer template.