Shuffling sequences with LSTM

2dm · July 30, 2021, 8:28pm

Hi,

I am running ppo and adding lstm to my model using the auto wrapper.
I just upgraded ray from 1.2.0 to 1.5.0 and I’m missing the warning about disabling the data shuffling: “Not time-shuffling RNN data for SGD”
I assumed that it is really for the best when using lstm and having time series inputs.

Should I explicitly change “shuffle_sequences” to False from now on, or is my assumption wrong to begin with?
Could you please explain why this change happened?

Thanks!

mannyv · July 31, 2021, 10:56am

Hi @2dm,

It looks like they just removed the warning message. The seems are still not being shuffled.

github.com

ray-project/ray/blob/96c69f8c77074cf9161aafe20bd5ba229a1dd436/rllib/utils/sgd.py#L65

    
      
              generator that returns mini-SampleBatches of size sgd_minibatch_size.
          """
          if not sgd_minibatch_size:
              yield samples
              return
          
          
if isinstance(samples, MultiAgentBatch):
              raise NotImplementedError(
                  "Minibatching not implemented for multi-agent in simple mode")
          
          
if "state_in_0" not in samples and "state_out_0" not in samples:
              samples.shuffle()
          
          
all_slices = samples._get_slice_indices(sgd_minibatch_size)
          data_slices, state_slices = all_slices
          
          
if len(state_slices) == 0:
              if shuffle:
                  random.shuffle(data_slices)
              for i, j in data_slices:
                  yield samples.slice(i, j)

Topic		Replies	Views
'use_lstm' wrapping in older and newer Ray versions RLlib	0	608	March 16, 2022
LSTM wrapper giving issue when used with trainer.compute_single_action RLlib	9	947	April 25, 2022
How are minibatches spliced RLlib	15	1391	November 11, 2021
How does `shuffle_sequences` work in PPO? RLlib	6	762	February 12, 2022
Custom model with LSTM crashes PPO sampler.py RLlib	0	250	November 24, 2023

Shuffling sequences with LSTM

Related topics