# When does an environment reset()?

**URL:** <https://discuss.ray.io/t/when-does-an-environment-reset/3729>\
**Category:** RLlib\
**Created:** [October 5, 2021, 5:48pm UTC](https://discuss.ray.io/t/when-does-an-environment-reset/3729 "2021-10-05T17:48:05Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ofir\_Abu](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/ofir_abu/32/650_2.png) [@Ofir\_Abu](https://discuss.ray.io/u/Ofir_Abu)\
**Post date:** [October 5, 2021, 5:48pm UTC](https://discuss.ray.io/t/when-does-an-environment-reset/3729/1 "2021-10-05T17:48:05Z")

</div>

Hi,  
Say I’m running some A3C agents on an arbitrary environment usign tune.run().  
When I see the reward over time steps graphs I see that there are a lot of timesteps (10M) but how do I know when the environment had been reset if at all?

If I don’t call env.reset() in my code - does the agents take steps in the environment until they’re done?

---

<div class="post-metadata">

**Author:** ![arturn](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/arturn/32/2096_2.png) [@arturn](https://discuss.ray.io/u/arturn)\
**Post date:** [October 5, 2021, 7:22pm UTC](https://discuss.ray.io/t/when-does-an-environment-reset/3729/2 "2021-10-05T19:22:49Z")

</div>

Hi @Ofir_Abu,

Generally an RLLib worker calls the reset() method of it’s environment instance after the environment returns a `done=True` from the `step()` call. The `dones` are part of the experiences that your workers produce. You can access these experiences and log them if you are interested. Otherwise they are accumulated in their own metrics: [episodes\_this\_iter](https://github.com/ray-project/ray/blob/957f9e9d99b2e06c69a5ecf3c19c4ece802ba83d/rllib/evaluation/metrics.py#L213) !

Cheers

---

<div class="post-metadata">

**Author:** ![mannyv](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/mannyv/32/606_2.png) [@mannyv](https://discuss.ray.io/u/mannyv)\
**Post date:** [October 5, 2021, 11:44pm UTC](https://discuss.ray.io/t/when-does-an-environment-reset/3729/3 "2021-10-05T23:44:04Z")

</div>

@Ofir_Abu there is also a horizon key in the config. If you set that to an integer n then rllib will artificially end the episode after that many steps of the environment, store a done=True, and call reset.

---

<div class="post-metadata">

**Author:** ![Archana\_R](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/archana_r/32/3829_2.png) [@Archana\_R](https://discuss.ray.io/u/Archana_R)\
**Post date:** [February 7, 2023, 8:55am UTC](https://discuss.ray.io/t/when-does-an-environment-reset/3729/4 "2023-02-07T08:55:10Z")

</div>

My code is as below

for \_ in range(1): ----------------------------------------------\> isnt this the episode number ?  
result = agent.train()

Since the episode number is 1 , shouldn’t the reset function be called ONCE ? For me the reset function is called 3-4 times in this 1 episode.

Batch size for debug purposes is 128 .

Can you please share your feedback ?

---

<div class="post-metadata">

**Author:** ![mannyv](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/mannyv/32/606_2.png) [@mannyv](https://discuss.ray.io/u/mannyv)\
**Post date:** [February 7, 2023, 11:49am UTC](https://discuss.ray.io/t/when-does-an-environment-reset/3729/5 "2023-02-07T11:49:39Z")

</div>

Hi @Archana_R,

Each call to train collects `train_batch_size` new steps from the environments.

It will reset the environment as many times as it needs to in order to collect that many steps.

It resets the environment every time it returns done or if horizon is set and it reaches that many steps in the episode.

---

<div class="post-metadata">

**Author:** ![Archana\_R](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/archana_r/32/3829_2.png) [@Archana\_R](https://discuss.ray.io/u/Archana_R)\
**Post date:** [February 7, 2023, 1:49pm UTC](https://discuss.ray.io/t/when-does-an-environment-reset/3729/6 "2023-02-07T13:49:45Z")

</div>

This helps. thanks !