# Controlling the exact number of episodes performed in a VectorEnv during a custom evaluation

**URL:** <https://discuss.ray.io/t/controlling-the-exact-number-of-episodes-performed-in-a-vectorenv-during-a-custom-evaluation/8219>\
**Category:** Configure Algorithm, Training, Evaluation, Scaling\
**Created:** [November 10, 2022, 2:33pm UTC](https://discuss.ray.io/t/controlling-the-exact-number-of-episodes-performed-in-a-vectorenv-during-a-custom-evaluation/8219 "2022-11-10T14:33:18Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![loicsacre](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/loicsacre/32/348_2.png) [@loicsacre](https://discuss.ray.io/u/loicsacre)\
**Post date:** [November 10, 2022, 2:33pm UTC](https://discuss.ray.io/t/controlling-the-exact-number-of-episodes-performed-in-a-vectorenv-during-a-custom-evaluation/8219/1 "2022-11-10T14:33:18Z")

</div>

I am trying to perform a custom evaluation with a [VectorEnv](https://docs.ray.io/en/latest/rllib/package_ref/env/vector_env.html) following the code given here:

> <https://github.com/ray-project/ray/blob/master/rllib/examples/custom_eval.py#L131-L135>

My vector env has `num_envs` sub cloned envs and my evaluation set is composed of `X` initial states of the environement. For instance, in the first episode a robot should grasp a red cube, in the second one a green cube, etc.

Let’s imagine that I have only have 1 rollout worker, `num_envs=4` and `X=20`, so each envs should performe `20/4=5` evaluation episodes.

**What would be the solution for performing exactly one episode on each sub env of a VectorEnv at once during a custom evaluation ?**

`w.sample.remote()` works well for a classic gym environment, even if there are several rollout workers.

I have deeply investigated the RLLib codebase to understand the implementation behind the `sample` method of the `RolloutWorker` but none of the combination of the configs `batch_mode` and `evaluation_duration_unit` would result in the desired behavior.

It is possible to have a more fine grained control over the evaluation loop ?

**Note** : I am trying to use a VectorEnv because an instance of [Isaac Sim](https://developer.nvidia.com/isaac-sim) env is very resource consuming. It is not possible to have more than two envs (one for sampling and one for evaluation) and Isaac Sim is optimized to handle several cloned envs in one main env (see some of their [OmniIsaacGymEnvs](https://github.com/NVIDIA-Omniverse/OmniIsaacGymEnvs) examples).

---

<div class="post-metadata">

**Author:** ![kourosh](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/kourosh/32/2130_2.png) [@kourosh](https://discuss.ray.io/u/kourosh)\
**Post date:** [January 5, 2023, 4:40pm UTC](https://discuss.ray.io/t/controlling-the-exact-number-of-episodes-performed-in-a-vectorenv-during-a-custom-evaluation/8219/2 "2023-01-05T16:40:49Z")

</div>

Hi @loicsacre,

What you want to achieve is a deterministic way too loop through states of the env. I have a couple of solution candidates which I am gonna give a highlevel description for.

1. You can override the evaluation logic entirely and introduce your own evaluation loop. See this: [ray/custom\_eval.py at master · ray-project/ray · GitHub](https://github.com/ray-project/ray/blob/master/rllib/examples/custom_eval.py)

2. You can use a callback’s `on_sub_environment_created()` or `on_episode_start()` (I am not sure exactly which of the top of my head) to set the initial state of your env. A callback can also be stateful which will help you to keep track of the states that have been used in a single process. However if you use more than 1 rolloutworker, you need to use the worker\_idx to determine which worker you are on to be able to loop through states that are not visited by other workers.