I can be wrong here, but I don’t think it is necessary to set shuffle_sequences
at all. It was not necessary for me
Did you set also the seed for the action_space in your environment as described in the end of the discussion here : Reproducible training - setting seeds for all workers / environments - #15 by Lauritowal
env.action_space.seed(RANDOM_SEED)