# Understanding SAC: Data Collection and Training

**URL:** <https://discuss.ray.io/t/understanding-sac-data-collection-and-training/11931>\
**Category:** RLlib\
**Created:** [August 24, 2023, 4:21pm UTC](https://discuss.ray.io/t/understanding-sac-data-collection-and-training/11931 "2023-08-24T16:21:48Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![JayCarrot](https://avatars.discourse-cdn.com/v4/letter/j/90ced4/32.png) [@JayCarrot](https://discuss.ray.io/u/JayCarrot)\
**Post date:** [August 24, 2023, 4:21pm UTC](https://discuss.ray.io/t/understanding-sac-data-collection-and-training/11931/1 "2023-08-24T16:21:48Z")

</div>

**How severe does this issue affect your experience of using Ray?**

- None: Just asking a question out of curiosity

* * *

Hi,

I’ve been working with PPO in RLlib, but I’m pretty new to off-policy ones, particularly the SAC.  
I’d like to learn about how the SAC algorithm collects data and processes training batches in the RLlib’s implementation.

Below, I’ve summarized what I know so far and outlined some questions I have on the topic.

  

In my understanding,

## SAC

- **Let’s assume:**

- **Data collection**

- **Train from batches**

## Questions

1. Are the descriptions in **Data collection** accurate? If not, please help me get it right.

2. Are the descriptions in **Train from batches** accurate? If not, I would appreciate any corrections or additional details.

3. Does the `training_intensity` always match the ratio of the number of _train\_op_ to the number of _data\_op_ carried out in an SAC iteration?  
e.g.

4. If the Q3 is correct, what happens when the ratio of `training_intensity` to “natural\_ratio” becomes a non-integer?

5. `min_sample_timesteps_per_iteration` and `min_time_s_per_iteration` are set to 100 and 1, respectively, _by default_? But in some examples `rollout_fragment_length=1` (with `num_wokers=1`, `num_envs_per_worker=1`), which could silently allow the rollout worker to collect 100 or more samples in an iteration. Why they are set to the values in the default SAC config?

6. In the training results, I think “num\_env\_steps\_trained\_this\_iter” shows doubled over what it was supposed to be, given the Q3 was accurate. Is this just because we train two network _types_ (i.e. Q net(s) and policy net)?